[SOLVED][Performance git-master] darkroom much less performant, dt-2.7 vs. dt-2.6.2

tempted but a bit scared to waste even more productive time… I have a backlog of images to process that will take months of continuous work (quite impractical for all purposes)…

@aadm Understand you very well. I also edited a lot with 2.7 and hence cannot go back, as 2.6.2 cannot read the XMP files Version 3

Me I’m fine now, as dt reached a level currently wich I can live with. Sometimes pain is necessary to move forward :relaxed:
I learned so a lot…

Also you could run 2.7 with - - config dir for trials :innocent:

I did some performance comparison between the recent dt-2.7 versions

…maybe someone is interested…

@pk5dark could you do me a favour and compare the instantaneous reaction between master and 2.6.2 when you grab the WB slider with the mouse and track it for and back? For me, 2.6.2 is still more responsive, means less lag in screen update…

I can’t promise that I will find time to do this as I need to get 2.6.2 running before. For me draging the WB slider in current git master is fast enough, but I played with the opencl settings before to get rid of the slugish behaviour. IRC opencl_sync_cache made a difference if changed.

Do you use the same opencl settings for your tests in darktablerc for 2.6.2 and 2.7.0?

@pk5dark
you are brilliant, that was it!
as 2.6.2 do not support (AFAIK) opencl_synch_cache=active module, so I set it there to opencl_synch_cache=false

Changing to false made the trick in 2.7

I wonder, should I issue a bug report? For this I have two suspicious topics:

  • just now I googled after “opencl_synch_cache=active module” and surprisingly I found redmine is still acvtive, I thought everything moved on to github
  • If I remember right, *
    just now I googled after “opencl_synch_cache=active module” got introduced somwhere on github, but with google I cannot find it anymore, otherwise I would just comment at that place…

I found it on github #2230

@pk5dark
in this respect, what did you mean with IRC?

So yes, setting it to opencl_synch_chache=active module costs some performance when editing something with a short pixlepipe (probably logic) but setting it to false is also not a solution. If one maybe wanna edit “levels” at the end of the que in such heavily edited pic.

Here “active module” really gains a lot (at least sometimes, in other times you need to switch the module off and on first)
:slight_smile:

So compiled 2.6.2 and running beside 2.7 gives me identical response time on moving the WB slider. Adding some instance of profiled denoise and equalizer makes no difference. Using retouch healing is slow in both versions. So no differences for me …

Should have been IIRC :innocent: (if i remember correctly)

@pk5dark Thank you very much!

Which opencl_synch_cache setting you used?

Hardware spec?
Here is core I9 9900K no OC, 32GB 3333MHz, 2x GTX1060

I have been lurking with interest. On inspecting my darktablerc, which I have never tweaked, I discovered:
opencl_sync_cache=false
and
opencl_omit_whitebalance=

I’d be happy to tweak away at any or all of the opencl settings, but is there documentation that gives a commentary on what possible values the individual options have, and - indeed - what the
values mean? For example:
opencl_device_priority=*/!0,*/*/*
opencl_enable_markesteijn=true

Again, the numeric values are probably defaults either for a generic hardware, or - possibly I guess - defaults chosen with the detected hardware in mind.

Many configuration files are peppered with comments - often the comments comprise 4 or 5 lines per setting. This one has not a single comment, nor a nod to a documentation file.

It is nice to have the popup bubble helptext when using the darktable preferences gui, although this has on occasion been cryptic. I’d love to see this level of commentary for all option lines in darktablerc.

@martin.scharnke

for opencl_synch_cache=active module (I ever wonder is sync nor synch?) AFAIK is new in 2.7 and you can find some things here on github

Other opencl tweaks are described here: usermanual/en/darktable_and_opencl_optimization and the following pages

That one is about having more than one GPU and you find things here: usermanual/en/darktable_and_opencl_multiple_devices

A quick googling on this also didn’t bring anything to me :slight_smile:

currently dt seems to have a function, where all masked entries in darktablerc will be moved to the top and the rest gets sorted, so they are in order.

I support your idea, in that case better keep the settings grouped.

Have you ever considered to file a feature request on github?

@pk5dark
I hope I did the right thing in filing this bug report

It is really not easy to describe, what I experiance there… :slight_smile:

system i9900k + GeForce GTX 1060 6GB

settings for 2.6.2 (default as I used a new config dir)

opencl_async_pixelpipe=false
opencl_avoid_atomics=false
opencl_checksum=2160716749
opencl_device_priority=*/!0,*/*/*
opencl_disable_drivers_blacklist=false
opencl_library=
opencl_mandatory_timeout=200
opencl_memory_headroom=300
opencl_memory_requirement=768
opencl_micro_nap=1000
opencl_number_event_handles=25
opencl_scheduling_profile=default
opencl_size_roundup=16
opencl_synch_cache=false
opencl_use_cpu_devices=false
opencl_use_pinned_memory=false

for 2.7

opencl_async_pixelpipe=FALSE
opencl_avoid_atomics=FALSE
opencl_checksum=2160716749
opencl_device_priority=0/!0,*/*/*
opencl_enable_markesteijn=true
opencl_library=
opencl_mandatory_timeout=200
opencl_memory_headroom=0
opencl_memory_requirement=768
opencl_micro_nap=100
opencl_number_event_handles=175
opencl_omit_whitebalance=
opencl_runtime=
opencl_scheduling_profile=very fast GPU
opencl_size_roundup=16
opencl_synch_cache=active module
opencl_use_cpu_devices=false
opencl_use_events=FALSE
opencl_use_pinned_memory=FALSE

Hmmmm…

interesting…:

  • our machines are quite comparable
  • we both use opencl_synch_cache=active module in dt 2.7
  • and you consider, your response is same than 2.6.2 when you move WB sliders (actually I have that lag also elsewhere), where both of us use opencl_synch_cache=false.

I thought that “active module” costs me some little performance and hence I opened that bug report, I listed above. Fortunately I said there already, it might be related to something totally different…

Below I have compared our both dt-2.7 darktablerc files.

  • you are not using async_pixelpipe, so I should be in benefint, but seems I am not
  • your priority setting is acc. to standard. you might try mine. Mine shall be exactly the correct one for multiple GPU
  • you use headroom=0, at least unusual :slight_smile:
  • event_handles, I use 300 and up to now I didn’t get that error, which is described in the handbook, but I could reduce to your value
  • strange is your setting opencl_omit_whitebalance= (and no value there)
  • The last four line are those, where is no equivalent in the other one’s and frankly speaking, after googling for such, I am the same stupid than before

What does all this teach us?
I’ll start from a virgin file again and merge my tweaks in that, let’s see.


Update ~5min later:
I did so, and except opencl_number_event_handles and opencl_async_pixelpipe it is more or less like my current one. So not that much learned up to now :-/


Your thoughts are very welcome…

AxelG   | opencl_async_pixelpipe=true
pk5dark | opencl_async_pixelpipe=FALSE
.............
AxelG   | opencl_scheduling_profile=multiple GPUs
AxelG   | opencl_device_priority=*/*/*/*

pk5dark | opencl_scheduling_profile=very fast GPU
pk5dark | opencl_device_priority=0/!0,*/*/*
.............
AxelG   | opencl_memory_headroom=300
pk5dark | opencl_memory_headroom=0
.............
AxelG   | opencl_number_event_handles=300
pk5dark | opencl_number_event_handles=175
.............
AxelG   | opencl_omit_whitebalance=false
pk5dark | opencl_omit_whitebalance=
.............
AxelG   | opencl_disable_drivers_blacklist=false
AxelG   | opencl=TRUE

pk5dark | opencl_runtime=
pk5dark | opencl_use_events=FALSE

There is still something very wrong with dt git master.
On the exact same empty (default) history stack, 2.6.2 looks nice and snappy, while git master is extremely laggy.
(and ui, even ignoring the theming changes, it looks comically misproportioned, oversized.)

The last revisions makes again darktable master fast on my computer as Axel or other confirmed (except for Axel the WB sliders as he himself told), and I didn’t see any differences by now with darktable 2.6.2, except lots of good adds that makes master a far better darktable ever.

About UI, yes you’re right, darktable.css adapt with default OS font make the UI comically misproportioned, oversized. That why darktable-elegant theme is here, to avoid that !

@Nilvus
for me it is not that drastically as @LebedevRI described, but I can still notice that.

It is important to say, I am not complaining. WIth those reports I want to support to make dt better!

I cannot call it very laggy, hence I issued below mentioned bug report.

Roman as you are a dev, maybe your voice counts more. Hence may I kindly ask you to look into this issue #2514 of mine?

Huh, it’s not default?

Nope, sorry, can’t do. I stopped contributing a year+ ago, except for basic camera support.
And seeing the progression since then, i wonder if i will have to move that too elsewhere.

Ok, I’ve written you said (except your last issue) that one of the last revision makes darktable now faster. And of course, darktable could, I think, be more faster anyway.

1 Like

No, darktable-elegant.css is not the default one. The default one is a version of this theme with OS default font and not Roboto, version of darktable-elegant.css made by @houz.

Depending of your OS font, that could change a lot the UI.

I don’t know what you don’t like in progression of darktable since a year but I think darktable just improves a lot since. Anyway, thanks for what you done on darktable and camera support since.