What does this output in Linux?

Reptorian · November 18, 2023, 5:04pm

For G’MIC on CLI users, let me know what this prints out for you:

 echo {`expr('11+x',256)`}

It looks like this in Windows:

♫☼►◄↕‼¶§▬${},←"↔▲▼ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~⌂ÇüéâäàåçêëèïîìÄÅÉæÆôöòûùÿÖÜ¢£¥₧ƒáíóúñÑªº¿⌐¬½¼¡«»░▒▓│┤╡╢╖╕╣║╗╝╜╛┐└┴┬├─┼╞╟╚╔╩╦╠═╬╧╨╤╥╙╘╒╓╫╪┘┌█▄▌▐▀αßΓπΣσµτΦΘΩδ∞φε∩≡±≥≤⌠⌡÷≈°∙·√ⁿ²■

I’d also like to know what does numbers before 10 shows up in Linux:

 echo {`[1]`} # Example

prawnsushi · November 18, 2023, 7:05pm

Got an error

[gmic]./ Start G'MIC interpreter (v.3.3.2).
[gmic]./run/__run/ *** Error *** Item substitution '{`expr(11+x,256)`}': Function 'expr()': First argument has invalid type 'scalar' (should be 'vector'), in expression 'expr(11+x,256)'.

Reptorian · November 18, 2023, 7:09pm

Can you execute it via user.gmic?

foo:
echo {`expr('11+x',256)`}

prawnsushi · November 18, 2023, 7:15pm

Sure.
Got 256 empty lines…
Well, 258 according to Kate.

Sorry, my bad, missed the “+x” part.

Here"s what i got :

${},123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~��������������������������������������������������������������������������������������������������������������������������������

Reptorian · November 18, 2023, 7:25pm

Thank you, a lot of missing characters.

I compared the result with UTF-16 data and this is what it looks like:

                                              ${},123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~��������������������������������������������������������������������������������������������������������������������������������
  ⊕⊖⊗⊘⊙∫∇  ≠√ ♠♣♥♦∩∪∧∨∈        € !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ ∞αβ∂δεηγλμωφπψρστθΔΣΓΩΦΠΨΘ←↑→↓↔↕ ¡¢£¤¥¦§¨©ª«¬®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ

Comparing this with Windows and your linux result:

                               ${},123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~��������������������������������������������������������������������������������������������������������������������������������

♫☼►◄↕‼¶§▬${},←"↔▲▼ !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~⌂ÇüéâäàåçêëèïîìÄÅÉæÆôöòûùÿÖÜ¢£¥₧ƒáíóúñÑªº¿⌐¬½¼¡«»░▒▓│┤╡╢╖╕╣║╗╝╜╛┐└┴┬├─┼╞╟╚╔╩╦╠═╬╧╨╤╥╙╘╒╓╫╪┘┌█▄▌▐▀αßΓπΣσµτΦΘΩδ∞φε∩≡±≥≤⌠⌡÷≈°∙·√ⁿ²

Looks like making different sort order for characters is going to be hard.

prawnsushi · November 18, 2023, 7:27pm

Is it somehow tied to locale settings?
I may be french but i use English locale on my computer.

Reptorian · November 18, 2023, 7:33pm

I think I’m going to ping @David_Tschumperle as I definitely would like some answers on this. I’ll do my own investigation on WSL on Windows.

prawnsushi · November 18, 2023, 7:39pm

Switched to fr_FR.UTF-8 (hopefully i did it right) and got the same results, it seems.
no èéà etc., even though i can type them anyway.

Reptorian · November 19, 2023, 12:33am

I got similar results with PowerShell after more testing. And I found out that the encoding used is UTF-8NoBom. Maybe it has to do with this, but I know nearly nothing about encoding. After looking around, I don’t see a solution here to this. However, G’MIC works as expected with the GUI interface with regards to text related things.

Ofnuts · November 19, 2023, 1:27am

Your output looks like good ol’ CP 437.

On Linux, terminal/CLI has moved to UTF-8 quite a while ago (because support for that useful € character). Control characters are very often ignored (as far as I can tell, only characters 8-13 have a visible & recognizable effect on the output.

 0: ----
 1: ----
 2: ----
 3: ----
 4: ----
 5: ----
 6: ----
 7: ----
 8: ---
 9: --  --
10: --
--
11: --
      --
12: --
      --
--: --
14: ----
15: ----
16: ----
17: ----
18: ----
19: ----
20: ----
21: ----
22: ----
23: ----
24: --▒--
25: ----
26: --▒--
27: ---
28: ----
29: ----
30: ----
31: ----
32: -- --

Reptorian · November 19, 2023, 1:53am

That gives me more answer to what’s going on. I also have this issue.

C:\WINDOWS\system32>gmic echo {`'Ç'`}
[gmic]./ Start G'MIC interpreter (v.3.3.3).
╟
[gmic]./ End G'MIC interpreter.

‘char’ is used to convert a string char into a numerical representation. The back tick tells G’MIC to to treat numbers as if they were characters instead of number. It gives me this.

No remedy unfortunately unless some one makes a really dedicated terminal for it.

In GUI plugin, code[local] and changing echo to status should reveal that it works as expected.

Edit: After some more knowledge, I have to conclude that G’MIC needs to remove UTF-16 characters for encoding/decoding and switch to UTF-8 replacement. There’s no terminal that supports UTF-16.

Here, some examples of UTF-8 character here - https://www.charset.org/utf-8 . Lots of them in fact.

Ofnuts · November 19, 2023, 9:54am

Could also be UCS-2, the difference being that UCS-2 can only encode 64K characters (all characters on 16-bits) while UTF-16 uses a mechanism similar to UTF-8 to encode more values.

David_Tschumperle · November 19, 2023, 11:03am

Some info about how the character encoding is managed in G’MIC.

G’MIC is actually encoding-agnostic, meaning that if you write a command in a file with a certain encoding, the result should depend on what encoding is used for output.
For instance, defining a command file like :

foo : 
  echo "ééé"

and invoking it in the terminal with gmic foo, should output ééé as long as the encoding used for the file and the terminal matches.
G’MIC just loads a command file as a binary file, so it does not really care about how the strings are encoded in the command file. All strings are actually stored as char* in the C++ code (so sequences of 8-bits, meaning a single “exotic” character can be encoded with several bytes, as in UTF-8 or UTF-16).

This being said, I don’t understand why

gmic echo {`'Ç'`}

won’t echo Ç at the end.
What does

gmic echo {'Ç'}

print ?

David_Tschumperle · November 19, 2023, 11:11am

For instance, on my Linux, my .gmic is encoded in UTF-8, and the command:

foo :
  e {'é'}
  e {`'é'`}

prints:

~$ gmic foo
[gmic]./ Start G'MIC interpreter (v.3.3.3).
195,169
é
[gmic]./ End G'MIC interpreter.

So, here, the é character is indeed encoded with 2 bytes: 195,169, a.k.a 0xC3,0xA9 in hexadecimal, which is coherent with what is indicated in the reference page

Reptorian · November 19, 2023, 4:00pm

It prints 199, and checking via Python, it works.

This works well after setting CMD to chcp 65001 which sets it to UTF-8 mode, and executing foo from file. The second line fails to print correctly directly from terminal.

Also, I get similar printing of chars to @prawnsushi after setting to UTF-8. Like 0 is missing and tons of missing characters.

In the case without G’MIC:

C:\Users\User>echo Ç
Ç

Reptorian · November 19, 2023, 4:52pm

I think I have my answer. I finally succeed in getting G’MIC to print characters correctly.

The answer is to have system-wide UTF-8 support.

C:\WINDOWS\system32>gmic echo {`'Ç'`}
[gmic]./ Start G'MIC interpreter (v.3.3.3).
Ç
[gmic]./ End G'MIC interpreter.

EDIT: Actually, never mind, something is still up:

C:\WINDOWS\system32>gmic echo ${rep_string_permutation_index2str\ §abcÁ×,10,1}
[gmic]./ Start G'MIC interpreter (v.3.3.3).
abcç
[gmic]./ End G'MIC interpreter.

Works here though:

C:\WINDOWS\system32>gmic echo {`'§abcÁ×'`}
[gmic]./ Start G'MIC interpreter (v.3.3.3).
§abcÁ×
[gmic]./ End G'MIC interpreter

There seems to be something very, very wrong with my own custom command, and I don’t understand it.

Here’s my guess to what happened. As @David_Tschumperle printed out
e {`'é'`} command, there is two numbers. My algorithm sorted the additional numbers. So, that’s why my code did not work out and prints out the wrong characters at the end. And that’s something I don’t know how to address.

My command does work on GUI plugin however, so this further complicates my debugging.