Exposure Fusion and Intel Neo drivers

[Entropy512 (Andy Dodd)]

@paperdigits

@Entropy512

As I already said, the commit in question has no description whatsoever, and does not reference the issue that drove it in any way, shape or form . No commit message, no comment in the code. One should not have to spend 20 minutes of Google searching to find something that should have been in the commit message or comments .

I agree it could be better, but it is what it is. Things are never perfect, we all do the best we can. Glad you found your answer.

@Entropy512

The developers immediately blamed the driver and blacklisted it across the board for all GPU variants and all possible driver versions, based on a single report - a report of a problem that turned out to be within darktable itself. (Failure to perform required cache invalidation of compiled kernels). The improper caching was solved by pull 2033 - yet the driver is still blacklisted.

Perhaps they just didn’t want to deal with it nor the bug reports that come with it. Good news it that it’s proper free software, so you can patch out the blacklisted module with ease.

I intend to do so.

@asn

You can overwrite it with: darktable --conf opencl_disable_drivers_blacklist=yes -d opencl

Let’s go back to the OP’s original comment:

@anon41087856

These users have been disregarded because, apparently, users are stupid and devs know better.

Here’s the user perspective:
Dartkable is working great
Hey, a new version of darktable!
Why is darktable now taking 4x as long for my workflow?

Most users are not going to do what I did and dig through git logs to figure out WHY darktable slowed down when they tried building from git. Even people who DO compile from source will likely initially assume they simply made a mistake when compiling by forgetting an important optional library (I did).

@anon41087856

Well, sometimes you just have to do it. There’s a lot of dynamic range out there, and it’ll be a long time before cameras will be able to cleanly comprehend every presentation of it in one exposure.

Just use masked dodging and burning, like in the old days in the darkroom. If it worked then, it should work better now.

Increasing the time I spend for a large number of images which I’ve exposed to preserve highlights from <20 seconds per image for a rough first pass to >10 minutes for something I’m happy with.

Entropy512:

The developers immediately blamed the driver and blacklisted it across the board for all GPU variants and all possible driver versions, based on a single report - a report of a problem that turned out to be within darktable itself. (Failure to perform required cache invalidation of compiled kernels). The improper caching was solved by pull 2033 - yet the driver is still blacklisted.

That’s out of topic. All tests of using Intel OpenCL we have done have failed, so the GPU codepath is not portable, and since dt is already optimized for CPU with SSE2 and SSE4/AVX/AVX2 support is incoming, there is no point investing effort to fix Intel OpenCL since it will not bring additionnal performance and does not prevent to use dt on CPU.

OK, I guess you opened a bit of a can of potential offtopic words with your comment in your first post which I quoted (otherwise I would not have responded to this topic)

Define “failed”. On an i5-7200U on Ubuntu 19.04 with the distribution-packaged Neo, I see a nearly 4x speedup from CPU to GPU with no discernible visual penalty. In the discussions of at least one pull request (the one where caching was fixed), not a single person indicated they had an issue with the Neo drives (at least on Linux) with the exception of the issues with cache invalidation.

dartkable_cpu_perf.txt (1.4 KB)

dartkable_neo_perf.txt (42.3 KB)

Entropy512:

Improper settings? OK, so - please tell me which settings of exposure EV and exposure bias in the exposure fusion mode of basecurve will not cause this:

There is one rule with base curve : don’t use base curve.

Then why does the module exist in darktable?

Entropy512:

What an average APS DSLR or MILC can do is irrelevant when you’re delivering to an sRGB display, other than making the exposure fusion approach described

That’s utterly wrong, since all you try to do is remap camera dynamic range to display dynamic range. RGB encode light emissions, you can’t treat them as black boxes of numbers. Please stop trolling out-of-topic and read https://medium.com/the-hitchhikers-guide-to-digital-colour

So the darktable team added a feature inspired by another program, specifically cited the program/algorithm, made a blog post about it - but failed to properly implement the algorithm in such a way that it’s impossible to get visual results even remotely resembling the algorithm claimed to be their inspiration.

Now the position is that not only should that feature not be fixed because obtaining visually acceptable results violates some rule of mathematical purity, but that feature shouldn’t even really exist in the first place nor should the module it was added to.

Anyone who points out the issues with said feature (despite the “slow and inefficient” original implementation of that feature working for them for years - the “slow and inefficient” approach being to export one JPEG, turn on the exposure module with +1 to +2 EV, export that, dupe the exposure module with same settings, export THAT, then feed to enfuse) is just a troll?

Given your statement made in your first post, you’re sounding REALLY hypocritical.

I apparently wasted my time by whipping up a patch to try and make my darktable workflow vastly more efficient by fixing a feature that doesn’t work as it was claimed to be intended to work. It’s pretty clear it won’t get accepted if I finish up the OpenCL side of things and attempt to submit, and any form of the patch that provides visually acceptable results will be rejected.