Proposal for the improvement of the reference documentation

afre · October 27, 2020, 5:48pm

@patdavid and co could help. We have lots of talent here.

I agree. Have fun!?

myselfhimself · October 28, 2020, 10:16am

A search field that helps filter out possibilities, searches and highlights in both function name and function content… and tolerates typographic mistakes thanks to levenshtein distance. Results would be command name + command description first letters.

This is showcased here: https://www.youtube.com/watch?v=iyj-Nyde-G4&feature=youtu.be

Sphinx (the doc generator for gmic-py) does some JS-based indexing and searching without any database instance up and I have seen in the past JS-levenshtein matchers already… For Ctrl+F is OK but it does not search in descriptions and does not tolerate mistakes…

myselfhimself · October 28, 2020, 10:19am

Old blog post on top JS fuzzy search libraries: 10 Awesome Javascript Fuzzy Search Libraries – Bashooka
I could find one for you if you miss time

Joan_Rake1 · October 28, 2020, 1:00pm

I like these grids. Alongside the old reference it’s pretty nice to have them broken down into categories, and I can find a few things I haven’t seen before…

David_Tschumperle · October 30, 2020, 4:31pm

Here is a small tutorial on how to generate the new (WIP) reference pages, for testing.
This requires several steps. The instructions below should work for any Linux distribution, as well as MSYS2 for Windows users.
The CLI tool gmic of G’MIC is used to generate the pages, so make sure it is already installed and available from your executable PATH.

Clone the git repo of the G’MIC project and activate the develop branch.

$ https://github.com/dtschump/gmic.git
$ cd gmic
$ git checkout develop

Go to the html folder, and create an empty test directory where all the generated files will be put.

$ cd gmic/html
$ mkdir test_ref
$ cd test_ref

Generate all the images for the reference documentation (beware, this takes some time!).

$ mkdir img
$ cd img  # Go to 'gmic/html/test_ref/img/'
$ gmic up parse_cli images

Generate the HTML files of the documentation (one file per command).

$ cd ..   # Go back to 'gmic/html/test_ref/'
$ gmic parse_cli html

Open your web browser, and go to URL : file://PATH_TO_GMIC/gmic/html/test_ref/categories.html.

You should be able then to browse the doc.
Note that this is still work in progress. Any feedback welcome. I’ve recently learnt how CSS works, and I’ve not used it a lot in the doc. This is gonna be improved in the future.

KaRo · October 31, 2020, 8:44pm

Works quite well, thank you.

Just a minor remark: for long custom function names word wrap and two liner boxes would be good for the overview.

David_Tschumperle · November 2, 2020, 2:30pm

I’ve made good progress on the doc generation today. I uploaded an updated version.

https://gmic.eu/reference2

KaRo · November 2, 2020, 2:31pm

Nice progress, with word-wrap etc.

Still there is in my build of the links one ‘html’ to much:

file:///Users/karo/sw/gmic/html/test_ref/animate3d.html.html#top

David_Tschumperle · November 2, 2020, 2:32pm

You’ll have to update the stdlib in one hour or so to fix this

afre · November 3, 2020, 4:30pm

To make it fun, let the user choose from a set of input images or integrate a simpler version of G’MIC online.

David_Tschumperle · November 4, 2020, 9:39am

My priorities right now:

Finalize the structure of the HTML reference documentation, and put it on the website to replace the older one.
Do the same for the .pdf version (which means I have to write a parse_cli_latex command, quite a pain).
At this point, I’ll be able to remove the document_gmic command that won’t be used anymore.
Write a post about the difference between stylize and fx_stylize for @lesolorzanov.
Then I’ll be able to start writing more detailed stuffs about the commands in the reference documentation.

Note that these are priorities concerning G’MIC, but I have of course other things that are inserted into this list, about my research work, the management of my research team, things to do for home and kids (xmas presents…). Sometimes I feel G’MIC is eating too much of my time.

afre · November 4, 2020, 4:10pm

Sometimes?
I can be lost in the moment as well.

KaRo · November 6, 2020, 2:58pm

Just had a look into the new documentation: really good, much better for me to find things!

Just one minor correction

Instead of “var$i” I would write “vari” or “var{i}”. One could read “var$i” is named “var[parameter i string]”.

David_Tschumperle · November 6, 2020, 3:33pm

About the doc: I’ve been working on it intensively the last days, and I think the improvements are really great. G’MIC has now its own light Markdown syntax that is used in the reference documentation and command description. It’s not as complete as the reference markdown, but at least it is entirely managed by G’MIC itself, so I don’t need external dependencies to generate the G’MIC documentation.
Hopefully, I’ll share more news when it’s ready.

David_Tschumperle · November 7, 2020, 5:43pm

Some news:

The new reference documentation of G’MIC is now ready and online : G'MIC - GREYC's Magic for Image Computing: A Full-Featured Open-Source Framework for Image Processing - Table of Contents
The .pdf version is missing, but I’ll try to generate it directly from the HTML pages in the future (rather than generating first a LaTeX output as I did before. I’ll see…).

At this point, if you want to contribute to improve the content of the documentation, you are welcome to do so
I’ve set up a markdown-like syntax in the G’MIC documentation generator, which can be used to decorate the description of the commands. This G’MIC markdown (a.k.a gmd) recognize the following patterns :

__words in italic__ \rightarrow words in italic
**words in bold** \rightarrow words in bold
`words in bold monospace` \rightarrow words in bold monospace (should be used for highlighting values or choices for arguments).
single_word_in_monospace' \rightarrow single_word_in_monospace (should be used for highlighting variable and command names).
''reference to page or command'' \rightarrow auto-link to URL describing the page or command (useful for the See also: subsections).
~~~code block~~~ \rightarrow

code block

^^ Image URL ^^ \rightarrow display image in a centered block.
{ list of values } \rightarrow highlight the list of values.
http[s]://address.url \rightarrow highlight and make a clickable link.
# Section name \rightarrow Start a new section.
## Subsection name \rightarrow Start a new subsection.
### Subsubsection name \rightarrow Start a new sub-subsection.
* Item of a list \rightarrow Display item as a list item.
- Item of a sub-list \rightarrow DIsplay item as a sub-list item.
. Item of a sub-sub-list \rightarrow Display item as a sub-sub-list item.

For now, there are not other syntax recognized, but I think this is already enough to make the reference documentation better looking than before. Maybe I’ll add other gmd-rules in the future, if there are needs for new stuffs.

I’m now planning to implement a gmd->ascii converter, to allow G’MIC displaying the gmd-formatted text on the console (when using the CLI tool gmic). The gmd->html command is already used for generating the HTML doc.

That’s all for now. As always, Feedback appreciated!

paperdigits · November 7, 2020, 6:47pm

If you need help generating PDF from HTML, ping me, we are doing exactly that for the new darktable docs.

myselfhimself · November 7, 2020, 11:06pm

congratulations!!!

David_Tschumperle · November 11, 2020, 5:06pm

Update:

I think I’m done with the reference documentation manager in G’MIC.

Now, we have a quite flexible system able to generate the technical reference documentation, based on two distinct commands: parse_cli and reference.

Command parse_cli generates the help about the use of one specific command (or a set of commands).
Command reference generates the help about the general G’MIC syntax (substitution mechanism, image terminology, grammar of the language and so on…).

These two commands are able to generate the documentation in the following formats:

VT100-colored ascii (for terminal output): this is the output format used by command help (global help) and help command_name, when invoked from the CLI tool gmic. The command help has been recoded at this occasion. Below a screenshot of what it renders, in the terminal:

HTML: this is the output format used to generate the reference pages on the G’MIC website : https://gmic.eu/reference . Sections and commands of the reference doc are generated as distinct pages. Here is how it looks:

man pages (roff format): this is similar to the ascii format, but with specific roff codes, so that it can be read with

$ man gmic

pdf: this is similar to the html output but in a single large .pdf file. The .pdf is available on the G’MIC website too, at : https://gmic.eu/reference/gmic_reference.pdf

The cool stuff with the new documentation manager is that it understands something I called gmd, for G’MIC Markdown, which is a light markdown-like syntax allowing to render the documentation with text decoration, images, cross-reference links, etc.

Note that at this point, I haven’t really improved the content of the documentation, just the way it is (automatically) generated, and the formatting features it supports.

Honestly, all this took me much more time than I thought at the beginning, but at least it’s there and I’ll be able to enrich the documentation more easily (I have a new command update_reference_html that generates and update the doc automatically on the G’MIC server, it could even be cron-ed if necessary).

As a result, I’ve removed the old doc generator (command document_gmic) that is no more of use.

Now, I’m thinking about taking a few dozen minutes each day, to write detailed documentation for each command. Any suggestion on with what command I should start?

KaRo · November 12, 2020, 3:11pm

Really nice progress.

Just one remark to pdf documentation. There are still links pointing to David internal locations of type file: e.g. “file:///home/dtschump/work/src/gmic/html/reference/examples_of_use.html#top”!

David_Tschumperle · November 12, 2020, 5:06pm

Fixed, thanks @KaRo!