Syntax Highlighting development for G'MIC

Reptorian · December 24, 2022, 8:37pm

Another followup to this thread: On Syntax Highlighting for G'MIC : Third thread.

I have managed to install my own G’MIC syntax highlighting file by installing a .xml file from KSyntaxHighlighting source code into appdata/local/org…/syntax folder. That means I am able to modify the xml file and see how it would apply in KDE Kate. I can’t say I have success with it, but I do have this .xml for KDE Kate which so far is slightly modified version of bash.xml.

gmic.zip (13.3 KB)

Feel free to contribute and improve the xml file. I will be doing this too.

garagecoder · December 24, 2022, 10:33pm

I will at least have a look… many years ago I did something similar for notepad++, but I think there was a syntax update which broke part of it. In other words, be ready to maintain it!

Reptorian · August 9, 2023, 4:22pm

Huge Update!

I finally got something here:

This is starting from scratch.

For KDE Kate:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE language SYSTEM "language.dtd"
[
    <!ENTITY commentary "^#( |$|(#)+).*">
    <!ENTITY cli-header "#@cli.*">
    <!ENTITY accessed-variables "\$(-\d+|\w+|\d+)">
    <!ENTITY command-names "\w+:">
]>

<language
    name="G'MIC"
    version=".01"
    kateversion="5.79"
    section="Development Syntax Highlighting"
    extensions="*.gmic"
    mimetype="application/x-shellscript"
    casesensitive="1"
    author="Miguel Lopez (reptillia39@live.com)"
    license="LGPL"
>
    
    <highlighting>
        <list name="control-flow">
            <item>apply_timeout</item>
            <item>check</item>
            <item>check3d</item>
            <item>do</item>
            <item>done</item>
            <item>if</item>
            <item>elif</item>
            <item>else</item>
            <item>fi</item>
            <item>error</item>
            <item>exec</item>
            <item>exec_out</item>
            <item>for</item>
            <item>foreach</item>
            <item>local</item>
            <item>mutex</item>
            <item>noarg</item>
            <item>onfail</item>
            <item>parallel</item>
            <item>progress</item>
            <item>quit</item>
            <item>repeat</item>
            <item>return</item>
            <item>rprogress</item>
            <item>run</item>
            <item>skip</item>
            <item>while</item>            
        </list>
        
        <list name="loop-control-flow">
            <item>continue</item>
            <item>break</item>           
        </list>
    
        <contexts>
            
            <context attribute="Normal Text" lineEndContext="#pop" name="BashOneLine" fallthroughContext="Command">
                <RegExpr attribute="Accessed-Variables" context="#stay" String="\$(-\d+|\w+|\d+)" beginRegion="Fold"/>
                <keyword attribute="Control Flow"  context="#stay" String="control-flow" />
            </context>
            
        </contexts>
        
        <itemDatas>
            <itemData name="Normal Text" defStyleNum="dsNormal" />
            <itemData name="Control Flow" defStyleNum="dsKeyword" />
            <itemData name="Accessed-Variables" defStyleNum="dsVariable" />
        </itemDatas>
    
    </highlighting>

</language>

Reptorian · August 9, 2023, 5:05pm

@grosgood

I’m trying to figure out regex string to match accessed variables. Here are some examples:

${1=2}
$*
$#
$1
$-1
${1-2}
${1--2}
$abc
$$f

What are more cases to test? Basically, I’d like to exclude ${1=?} case or the first one. Basically a regex for every accessible variables.

Reptorian · August 9, 2023, 7:15pm

Here are more progress on syntax highlighting. Still some issues.

EDIT:

I did what I could, still not finished:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE language SYSTEM "language.dtd"
[
]>

<language
    name="G'MIC"
    version=".01"
    kateversion="5.79"
    section="Development Syntax Highlighting"
    extensions="*.gmic"
    mimetype="application/x-shellscript"
    casesensitive="1"
    author="Miguel Lopez (reptillia39@live.com)"
    license="LGPL"
>
    
    <highlighting>
        <list name="control-flow">
            <item>apply_timeout</item>
            <item>check</item>
            <item>check3d</item>
            <item>do</item>
            <item>done</item>
            <item>if</item>
            <item>elif</item>
            <item>else</item>
            <item>fi</item>
            <item>exec</item>
            <item>exec_out</item>
            <item>for</item>
            <item>foreach</item>
            <item>l</item>
            <item>local</item>
            <item>mutex</item>
            <item>noarg</item>
            <item>onfail</item>
            <item>parallel</item>
            <item>progress</item>
            <item>quit</item>
            <item>repeat</item>
            <item>return</item>
            <item>rprogress</item>
            <item>run</item>
            <item>skip</item>
            <item>while</item>            
        </list>
        
        <list name="loop-control-flow">
            <item>continue</item>
            <item>break</item>
        </list>
    
        <contexts>
            
            <context attribute="Normal Text" lineEndContext="#pop" name="BashOneLine" fallthroughContext="Command">
                <RegExpr attribute="CLI Header" context="#stay" String="#@cli.*" />
                <RegExpr attribute="Command Names" context="#stay" String="^\+?([a-zA-Z0-9_]*)+\s?:( |$)" />
                <RegExpr attribute="Accessed-Variables" context="#stay" String="(?&lt;!^#)(?&lt;!#\$)\$\$??(\{)?(?(1)-?\d+--?\d+(:\d+}|})|(-?\d+|\w+|!|&gt;|&lt;|\/|\||\*|#|\^))" />
                <RegExpr attribute="Comments" context="#stay" String="(\w|^|\s)(#\w+$|#\s+.*|#+$|#--+.*|#@gmic)"/>
                <keyword attribute="Control Flow"  context="#stay" String="control-flow" />
                <keyword attribute="Loop Control Flow"  context="#stay" String="loop-control-flow" />
                <RegExpr attribute="Error" context="#stay" String="\berror\b" />
            </context>
            
        </contexts>
        
        <itemDatas>
            <itemData name="Normal Text" defStyleNum="dsNormal" />
            <itemData name="CLI Header" defStyleNum="dsDocumentation" />
            <itemData name="Command Names" defStyleNum="dsFunction" />
            <itemData name="Accessed-Variables" defStyleNum="dsVariable" />
            <itemData name="Comments" defStyleNum="dsComment" />
            <itemData name="Control Flow" defStyleNum="dsKeyword" />
            <itemData name="Loop Control Flow" defStyleNum="dsAlert" />
            <itemData name="Error" defStyleNum="dsError" />
        </itemDatas>
    
    </highlighting>

</language>

Use it in KDE Kate however you please.

The final thing I want to do is to highlight every section of code between quotation mark, and that factors into escape character. And doing this while preserving highlights of other code. I don’t know how to do that, but that’s the bare minimum.

Reptorian · August 10, 2023, 7:41pm

More update:

I think I’m figuring out how stay and pop works. Still not there.

If you need it, here’s the current code:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE language SYSTEM "language.dtd"
[
]>

<language
    name="G'MIC"
    version=".01"
    kateversion="5.79"
    section="Development Syntax Highlighting"
    extensions="*.gmic"
    mimetype="application/x-shellscript"
    casesensitive="1"
    author="Miguel Lopez (reptillia39@live.com)"
    license="LGPL"
>
    
    <highlighting>
        <list name="control-flow">
            <item>apply_timeout</item>
            <item>check</item>
            <item>check3d</item>
            <item>do</item>
            <item>done</item>
            <item>if</item>
            <item>elif</item>
            <item>else</item>
            <item>fi</item>
            <item>exec</item>
            <item>exec_out</item>
            <item>for</item>
            <item>foreach</item>
            <item>l</item>
            <item>local</item>
            <item>mutex</item>
            <item>noarg</item>
            <item>onfail</item>
            <item>parallel</item>
            <item>progress</item>
            <item>quit</item>
            <item>repeat</item>
            <item>return</item>
            <item>rprogress</item>
            <item>run</item>
            <item>skip</item>
            <item>while</item>            
        </list>
        
        <list name="loop-control-flow">
            <item>continue</item>
            <item>break</item>
        </list>
    
        <contexts>
            
            <context attribute="Normal Text" lineEndContext="#pop" name="BashOneLine" fallthroughContext="Command">
                <RegExpr attribute="CLI Header" context="#stay" String="#@cli.*" />
                <RegExpr attribute="Command Names" context="#stay" String="^\+?([a-zA-Z0-9_]*)+\s?:( |$)" />
                <RegExpr attribute="Accessed-Variables" context="#stay" String="(?&lt;!^#)(?&lt;!#\$)\$\$??(\{)?(?(1)-?\d+--?\d+(:\d+}|})|(-?\d+|\w+|!|&gt;|&lt;|\/|\||\*|#|\^))" />
                <RegExpr attribute="Comments" context="#stay" String="(\w|^|\s)(#\w+$|#\s+.*|#+$|#--+.*|#@gmic)"/>
                <keyword attribute="Control Flow"  context="#stay" String="control-flow" />
                <keyword attribute="Loop Control Flow"  context="#stay" String="loop-control-flow" />
                <RegExpr attribute="Error" context="#stay" String="\berror\b" />
                <DetectChar attribute="Quotation Mark" context="string" char="&quot;" />
            </context>
            
            <context attribute="Quotation Mark" lineEndContext="#stay" name="string" >
                <DetectChar attribute="Quotation Mark" context="#pop" char="&quot;" />
            </context>
            
        </contexts>
        
        <itemDatas>
            <itemData name="Normal Text" defStyleNum="dsNormal" />
            <itemData name="CLI Header" defStyleNum="dsDocumentation" />
            <itemData name="Command Names" defStyleNum="dsFunction" />
            <itemData name="Accessed-Variables" defStyleNum="dsVariable" />
            <itemData name="Comments" defStyleNum="dsComment" />
            <itemData name="Control Flow" defStyleNum="dsKeyword" />
            <itemData name="Loop Control Flow" defStyleNum="dsAlert" />
            <itemData name="Error" defStyleNum="dsError" />
            <itemData name="Quotation Mark" defStyleNum="dsString" />
        </itemDatas>
    
    </highlighting>

</language>

Reptorian · August 11, 2023, 3:09am

Ok, I finally figured out how it works. Here’s stacked coloring:

Keep in mind there’s going to be limitation because Kate does not support multi-line regex highlighting, and therefore I can’t idiot-proof it entirely or make it fool-proof. Oh well. I went to suggesting that idea to the Kate team.

EDIT: And I ran into a breaking bug that makes this impossible to finish with current Kate limitation. Cancelling. It’s on Kate end.

This is where I found that current KDE Kate limitation is unacceptable:

grosgood · August 11, 2023, 11:34am

Sorry for the late response. The weekend is upon me; I can now spend some time on this.
I don’t know Kate, if I may, I’d just like to write regular expressions for use by Python’s re package.
I guess you can transliterate to Kate starting from that.
I see you already match de-references like $[a-zA-Z0-9_]+ (they are cyan in your screen grabs). You need the same but bracketed in curly braces as well {}. However, if = appears anywhere within {} with at least [0-9]+ on the left of = and [0-9]* on the right, you don’t want to match that.

Reptorian · August 11, 2023, 5:54pm

Yes, and it turns out that the regex to fit the criteria is this: (?<!^#)(?<!#\$)\$\$??(\{)?(?(1)-?\d+--?\d+(:\d+}|})|(-?\d+|\w+|!|>|<|\/|\||\*|#|\^)). That will give me non-local command accessed variables.

That being said, I am finding out that it seems that syntax highlighting for G’MIC is more looking like a insanely complex problem.

afre · August 12, 2023, 2:26am

2 posts were merged into an existing topic: WIP: Unified Image VIewer for G’MIC