Thanks!
Had some fun with this. However, I’m putting the topic of threads off to the tail end of the Mathematical Expression tutorials. For that audience, people coming up the curve on G’MIC, the sanest advice I can (and should) give is: “Let G’MIC choose the threading model. Period. Full Stop.”
Oh, yes, some words about threading prefixes ":", "*", ">" and "<", because people will see them lurking in code (“What are those?”), but those words will come with caveats: ‘*’ and ‘:’ may not actually speed up processing at all. Likewise, don’t use ">" or "<" unless you know your particular algorithm demands a visitation order. Not sure? Then don’t use either, as guaranteed visitation order requires single-threaded operation, which will certainly be less optimal.
But — you’re past that audience. Have all the fun with threads you want.
I rewrote your script a bit for investigative purposes:
rmerge.gmic
1 rep_maximum_average: -skip ${1=""},${2=""}
2 # textype ⇒ *t*hread *ex*ecution *t*ype
3 textype,textypetoo=${1-2}
4 number_of_threads:=$_cpus
5 echo "We\ have:\ "$number_of_threads
6 echo "Our\ thread execution types\ -\ input:\ "$textype",\ eval:\ "$textypetoo
7
8 {$number_of_threads*100},{$number_of_threads*100},1,3,$textype${}"P = vector3(0);
9 idx = round(3*t/$number_of_threads,-1);
10 P[idx]= round(u(11*t/$number_of_threads,11,1,1),-1);
11 P
12 "
13 -name. noiseimage
14 +normalize[noiseimage] 0,255
15 -output. mthreads.png
16 -remove.
17
18 -eval[noiseimage] $textypetoo${}"begin_t(
19 avg_values_per_thread=0;
20 print(t)
21 );
22 avg_values_per_thread+=norm(I);
23 end_t(
24 critical(print(t,avg_values_per_thread/100^2*$number_of_threads));
25 );
26 end(
27 merge(avg_values_per_thread,max);
28 print('---max_average---');
29 print(avg_values_per_thread/100^2*$number_of_threads)
30 )
31 "
32 -split[noiseimage] y,$number_of_threads
33 repeat $!
34 local[$>]
35 -channels. {int(3*$>/$number_of_threads)}
36 -echo {$>+1}":\ Average:\ "{ia}",\ Variance:\ "{iv}",\ Minimum:\ "{im}",\ Maximum:\ "{iM}",\ Sum:\ "{is}
37 -done
38 -done
39 -remove
Line 18: To make per-thread instances of avg_values_per_thread
you need to use begin_t()
instead of just begin()
— that unadorned version runs once in thread 0; you get only one instance of avg_values_per_thread,
one that only thread 0 populates. You have no other instances from which to choose a maximum in your call to merge()
. begin_t()
does per-thread set-up. Variables defined in that call are local to the thread. Now you have per-thread instances of avg_values_per_thread
from which you can merge values.
Line 23: Likewise, use end_t()
to do things just before the thread exits — like print out the bucket. See next remark.
Line 24: critical()
ensures that its expression argument runs atomically — that it doesn’t get split across different threads of execution. I used it because otherwise, the printing of the thread id could occur earlier or later than its associated datum — there would be no way to disambiguate the printout otherwise.
Overall: It generates a diagnostic image, mthreads.png
. The lower thread ids populated the red channel, middle thread id’s green, high thread id’s blue. Gives a quick visual confirmation that G’MIC assigns threads to sections of the pixel buffer in sequential order. Otherwise the colors would scatter all along the height of the image.
Dump of `rmerge.gmic` output
gosgood@bertha /media/lydia/mthreads $ gmic rmerge.gmic rep_maximum_average '*','*'
[gmic]-0./ Start G'MIC interpreter.
[gmic]-0./ Input custom command file 'rmerge.gmic' (1 new, total: 4596).
We have: 40
Our thread execution types - input: *, eval: *
[gmic_math_parser] t = (uninitialized) (mem[17]: scalar)
[gmic_math_parser] t = (uninitialized) (mem[17]: scalar)
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = (uninitialized) (mem[42]: scalar)
[gmic_math_parser] '---max_average---' = (uninitialized) (mem[44]: vector17)
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = (uninitialized) (mem[62]: scalar)
[gmic_math_parser] t = 4
[gmic_math_parser] t = 33
[gmic_math_parser] t = 11
[gmic_math_parser] t = 2
[gmic_math_parser] t = 23
[gmic_math_parser] t = 35
[gmic_math_parser] t = 21
[gmic_math_parser] t = 39
[gmic_math_parser] t = 13
[gmic_math_parser] t = 29
[gmic_math_parser] t = 7
[gmic_math_parser] t = 5
[gmic_math_parser] t = 27
[gmic_math_parser] t = 0
[gmic_math_parser] t = 19
[gmic_math_parser] t = 9
[gmic_math_parser] t = 31
[gmic_math_parser] t = 3
[gmic_math_parser] t = 25
[gmic_math_parser] t = 37
[gmic_math_parser] t = 15
[gmic_math_parser] t = 17
[gmic_math_parser] t = 36
[gmic_math_parser] t = 24
[gmic_math_parser] t = 26
[gmic_math_parser] t = 14
[gmic_math_parser] t = 18
[gmic_math_parser] t = 28
[gmic_math_parser] t = 10
[gmic_math_parser] t = 38
[gmic_math_parser] t = 6
[gmic_math_parser] t = 1
[gmic_math_parser] t = 20
[gmic_math_parser] t = 12
[gmic_math_parser] t = 32
[gmic_math_parser] t = 30
[gmic_math_parser] t = 8
[gmic_math_parser] t = 34
[gmic_math_parser] t = 22
[gmic_math_parser] t = 16
[gmic_math_parser] t = 15
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 30332.853667387008
[gmic_math_parser] t = 2
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 31002.581126758574
[gmic_math_parser] t = 23
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 46199.4686227684
[gmic_math_parser] t = 39
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 51480.988836662291
[gmic_math_parser] t = 13
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 35193.848779037406
[gmic_math_parser] t = 37
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 47521.786290954587
[gmic_math_parser] t = 20
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 40270.378002334597
[gmic_math_parser] t = 1
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 29007.301944291117
[gmic_math_parser] t = 24
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 48178.509871242524
[gmic_math_parser] t = 18
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 36297.851349411962
[gmic_math_parser] t = 8
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 42899.125388774875
[gmic_math_parser] t = 32
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 37610.50768864822
[gmic_math_parser] t = 17
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 34294.77197567463
[gmic_math_parser] t = 34
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 41575.587466009136
[gmic_math_parser] t = 12
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 50819.777767223357
[gmic_math_parser] t = 36
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 45535.799253311161
[gmic_math_parser] t = 14
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 28353.230868729115
[gmic_math_parser] t = 26
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 43341.428201053262
[gmic_math_parser] t = 38
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 49498.611350254061
[gmic_math_parser] t = 6
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 38946.776838678357
[gmic_math_parser] t = 16
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 32343.903074274062
[gmic_math_parser] t = 4
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 34964.318173113825
[gmic_math_parser] t = 30
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 33646.90011281109
[gmic_math_parser] t = 10
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 46861.159746437072
[gmic_math_parser] t = 28
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 29678.561232178214
[gmic_math_parser] t = 29
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 31666.907326784611
[gmic_math_parser] t = 7
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 40924.416886343002
[gmic_math_parser] t = 11
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 48839.458113040928
[gmic_math_parser] t = 31
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 35636.565902988434
[gmic_math_parser] t = 9
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 44877.175599760056
[gmic_math_parser] t = 3
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 33000.286287980081
[gmic_math_parser] t = 25
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 50159.828297389984
[gmic_math_parser] t = 35
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 43562.835854822159
[gmic_math_parser] t = 5
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 36957.409204721451
[gmic_math_parser] t = 27
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 27695.330300394893
[gmic_math_parser] t = 33
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 39615.845659919738
[gmic_math_parser] t = 19
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 38288.510858316426
[gmic_math_parser] t = 21
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 42232.88877002144
[gmic_math_parser] t = 0
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 27046.000944480351
[gmic_math_parser] t = 22
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 44219.414029459003
[gmic_math_parser] '---max_average---' = [ 45,45,45,109,97,120,95,97,118,101,114,97,103,101,45,45,45 ] (size: 17)
[gmic_math_parser] avg_values_per_thread/100^2*$number_of_threads = 51480.988836662291
1: Average: 5.5000457994219989, Variance: 10.090985759256853, Minimum: 2.612103708088398e-05, Maximum: 10.999990463256836, Sum: 2200018.319768799
2: Average: 5.6332893542196603, Variance: 9.5816783616372501, Minimum: 0.27504423260688782, Maximum: 10.999988555908203, Sum: 2253315.7416878641
3: Average: 5.7704154366585616, Variance: 9.1101382664903614, Minimum: 0.55001521110534668, Maximum: 10.999998092651367, Sum: 2308166.1746634245
4: Average: 5.9058766468685864, Variance: 8.6284305938681545, Minimum: 0.82500624656677246, Maximum: 10.999978065490723, Sum: 2362350.6587474346
5: Average: 6.0444772094285488, Variance: 8.1678953259097913, Minimum: 1.1000055074691772, Maximum: 10.999971389770508, Sum: 2417790.8837714195
6: Average: 6.1792098588848114, Variance: 7.7155750463362587, Minimum: 1.3750025033950806, Maximum: 10.999997138977051, Sum: 2471683.9435539246
7: Average: 6.3254137017980216, Variance: 7.286496033518346, Minimum: 1.6500345468521118, Maximum: 10.999953269958496, Sum: 2530165.4807192087
8: Average: 6.4534099814340475, Variance: 6.8637419200600469, Minimum: 1.9251043796539307, Maximum: 10.999983787536621, Sum: 2581363.9925736189
9: Average: 6.5977895209920403, Variance: 6.4495162883693062, Minimum: 2.2000439167022705, Maximum: 10.999987602233887, Sum: 2639115.8083968163
10: Average: 6.7406175768142935, Variance: 6.0609722856101138, Minimum: 2.4750003814697266, Maximum: 10.999974250793457, Sum: 2696247.0307257175
11: Average: 6.8765323236149554, Variance: 5.6664685828301025, Minimum: 2.7500076293945312, Maximum: 10.999983787536621, Sum: 2750612.929445982
12: Average: 7.0080290295583012, Variance: 5.3047347322104201, Minimum: 3.0250067710876465, Maximum: 10.999980926513672, Sum: 2803211.6118233204
13: Average: 7.144751217333674, Variance: 4.9398406505223793, Minimum: 3.3000013828277588, Maximum: 10.99998950958252, Sum: 2857900.4869334698
14: Average: 7.2814522379046682, Variance: 4.6009308417235362, Minimum: 3.575042724609375, Maximum: 10.999974250793457, Sum: 2912580.8951618671
15: Average: 7.426495402957797, Variance: 4.2626344751538436, Minimum: 3.8500277996063232, Maximum: 10.999984741210938, Sum: 2970598.1611831188
16: Average: 7.5649060485053061, Variance: 3.9432156546258401, Minimum: 4.1250200271606445, Maximum: 10.999997138977051, Sum: 3025962.4194021225
17: Average: 7.6947556419193743, Variance: 3.6237321491967598, Minimum: 4.4000482559204102, Maximum: 10.999964714050293, Sum: 3077902.2567677498
18: Average: 7.8387190625262262, Variance: 3.3337733210261691, Minimum: 4.6750025749206543, Maximum: 10.999994277954102, Sum: 3135487.6250104904
19: Average: 7.9730926009595393, Variance: 3.0407411731600331, Minimum: 4.9500021934509277, Maximum: 10.999984741210938, Sum: 3189237.0403838158
20: Average: 8.1185076229619977, Variance: 2.7872165787039629, Minimum: 5.2250041961669922, Maximum: 10.999999046325684, Sum: 3247403.0491847992
21: Average: 8.2501353002524382, Variance: 2.5244646324931344, Minimum: 5.5000343322753906, Maximum: 10.999993324279785, Sum: 3300054.120100975
22: Average: 8.3912606142354011, Variance: 2.2716497970423486, Minimum: 5.7750020027160645, Maximum: 10.999983787536621, Sum: 3356504.2456941605
23: Average: 8.5275903369712829, Variance: 2.0400438008999822, Minimum: 6.050015926361084, Maximum: 10.999996185302734, Sum: 3411036.1347885132
24: Average: 8.6589096027576922, Variance: 1.824979048672003, Minimum: 6.3250079154968262, Maximum: 10.99997615814209, Sum: 3463563.8411030769
25: Average: 8.798242226526737, Variance: 1.6123830444338805, Minimum: 6.6000075340270996, Maximum: 10.999978065490723, Sum: 3519296.8906106949
26: Average: 8.9384036519789696, Variance: 1.4218627859915545, Minimum: 6.8750162124633789, Maximum: 10.999991416931152, Sum: 3575361.4607915878
27: Average: 9.0753074894785879, Variance: 1.2314130959905678, Minimum: 7.1500191688537598, Maximum: 10.999990463256836, Sum: 3630122.9957914352
28: Average: 9.2130612678062924, Variance: 1.068267788592884, Minimum: 7.4250054359436035, Maximum: 10.999933242797852, Sum: 3685224.5071225166
29: Average: 9.3487650111269947, Variance: 0.90712310176046074, Minimum: 7.7000036239624023, Maximum: 10.999997138977051, Sum: 3739506.004450798
30: Average: 9.4864084709167482, Variance: 0.76263914503051577, Minimum: 7.9750089645385742, Maximum: 11, Sum: 3794563.3883666992
31: Average: 9.6247010512757303, Variance: 0.6295941643023546, Minimum: 8.2500114440917969, Maximum: 10.999995231628418, Sum: 3849880.4205102921
32: Average: 9.7635583670377724, Variance: 0.51059804363528893, Minimum: 8.5250129699707031, Maximum: 10.999996185302734, Sum: 3905423.3468151093
33: Average: 9.8999654232096663, Variance: 0.40186338801668781, Minimum: 8.8000068664550781, Maximum: 10.999999046325684, Sum: 3959986.1692838669
34: Average: 10.037592641599179, Variance: 0.30859681806450318, Minimum: 9.0750026702880859, Maximum: 10.999994277954102, Sum: 4015037.0566396713
35: Average: 10.174010604717731, Variance: 0.22740174993573165, Minimum: 9.3500022888183594, Maximum: 10.999998092651367, Sum: 4069604.2418870926
36: Average: 10.313058074333668, Variance: 0.1578218503463861, Minimum: 9.6250038146972656, Maximum: 10.999997138977051, Sum: 4125223.2297334671
37: Average: 10.449563414857387, Variance: 0.10088976044344576, Minimum: 9.9000053405761719, Maximum: 10.999998092651367, Sum: 4179825.365942955
38: Average: 10.587271196677685, Variance: 0.056689892499643783, Minimum: 10.175004959106445, Maximum: 10.999998092651367, Sum: 4234908.4786710739
39: Average: 10.725526492979526, Variance: 0.025144694210397604, Minimum: 10.450002670288086, Maximum: 11, Sum: 4290210.5971918106
40: Average: 10.862820333256721, Variance: 0.0063182894998874799, Minimum: 10.725000381469727, Maximum: 11, Sum: 4345128.1333026886
[gmic]-0./ End G'MIC interpreter.
Search for ---max_average---
in the dump. That locates the largest avg_values_per_thread
that merge(avg_values_per_thread,max)
picked: 51480.988836662291, accumulated by thread t = 39
. No surprise there. The largest thread id operated over the last portion in the buffer with minimum pixel values of ≈10.725 → 11.000. merge()
found this maximum value among all the thread-local copies of avg_values_per_thread
and picked it, as expected.
Small aside:
round(lerp(u(0,11,1,1),11,t/$number_of_threads));
This seems simpler:
round(u(11*t/$number_of_threads,11,1,1),-1);
All that you want to do is to move the lower boundary of the selection interval as a function of thread_id: larger id ⇒ smaller intervals. Just make the first argument of u()
such a function. Home free.
Have fun!