also see short discussion here vkdt dev diary, pt2 - #168 by g-man
1 Like
To be clear, the Feature Request is a joke. I don’t love the idea of ai for images. At some point I think ai manipulation becomes art and not a photo anymore.
1 Like
Whose art, though?
this is not in any way artificially intelligent. creating these 5d radiance fields from a couple of images is an impressive funny application. but in the end it’s some kind of continuous data approximation. the authors also write that at the end in the “ethical issues” section (or what it’s called). it does not incorporate data from any other images that might potentially contain faces of people that aren’t actually in the shot.
and as i suspected, most of the difference to stock methods is the black point handling. this has severe impact even for single raw files. see this example vkdt render (don’t have a darktable comparison here, sorry, but there we needed to jump some crazy hoops to get the black noise right):
(shaky night time raw image stopped up +5ev)
squinting at the noisy image you can imagine that any average only considering the positive/visible part of the pixel values here will give you wrong results. i’ll consider this well-known.
“AI” is a better solution to denoising and highlight reconstruction compare to traditional algorithms, and the reason is not mysterious : when we look at a noisy image of a face we can easily tell the noise from the signal because we know what faces look like, and the same is true of deep-learning models. Have a strong prior and understanding of what the image should look like is what you need to get better performances.
Personally I would love to have an open source alternative to commercial softwares, but I’m not sure there’s even good datasets available. For denoising we need a large collection of raw images shot at different ISO (potentially a model could be built on that to generate artificial noise so single shot image could be used instead).
you know this one right? Exporting/importing photos in full 32-bit representation (to train a neural network for image denoising)
and yes, the paper in question here is not “AI” in that sense, it hasn’t seen anything other than the stack of images to use for denoising (no strong prior from other images of the same subject, say).
2 Likes
I use DxO Photolab with Deep Prime noise reduction and can attest to how well it works. I have one particularly challenging example where I took a photo with a Canon APS-C camera at ISO 6400 that was still underexposed by almost two stops. Normal noise reduction in ART made it usable, but just barely, whereas with DxO I never would have guessed it was taken in such low light. Here are the two versions:
ART:
DxO:
In my experience, it does a great job of reconstructing the detail that is obscured by noise without creating artifacts. Yes, an extreme example like above will still have a bit of a waxy look to it (because additional detail was simply not present in the RAW file), but it still looks much better than what I could do with traditional methods.
I prefer a native Linux RAW editor in general, but I still fire up the ol’ Windows VM with Photolab for high ISO shots and use it basically as a preprocessor for ART. But that’s tedious.
2 Likes
You can use this (as mentioned by @hanatos above) relatively easily from within ART if you like, by writing a proper user command. Despite all the disclaimers and to-dos, I found it to be pretty amazing. The major downside is that it’s slow, especially if you don’t have a Nvidia GPU and run it on a mobile CPU like myself. But for the few times I needed it, it’s worth the wait imho. And dxo is not exactly fast either as far as I understood.
HTH
3 Likes
Very interesting! I’ll definitely give that a try. And yes, DxO Deep Prime is very slow, especially on a VM, but it only does the work on export, so I just walk away and do something else for a while.
This looks interesting…sounds like you have used it and are impressed…seems like there is a lua script for DT…may have to try and get that to run…
It would be neat to see a before and after that you have done just for demonstration…
1 Like
I gave it a try with the images I posted above. First I did some processing on the original RAW image in ART, but no noise reduction or sharpening, and exported as 16-bit TIF. Then I ran the nind-denoise script on it. It’s not bad but also not nearly as good as DxO. I think I actually prefer the version done in ART with regular noise reduction (posted above) because of the artifacts this has produced on the shadow areas of the face. Maybe there are settings to tweak that could improve it.
Original (downsized to 2000 pixels width):
Denoised:
1 Like
One issue with the approach is that it’s very hard to get pictures of people (and even more so of birds & animals) at different ISO, since they move. But the denoiser needs to be really good with people since we’re much better at seeing artifacts in them.
One solution could be to first train a network that can add noise to images (which presumably should be easier) so that we can generate training samples from single low-ISO shots.
Would you be able to share the raw? I have a different experience, if anything the results I get are too clean / “waxy” for my taste. Here are a couple of examples from playraws:
Thanks. Here’s what I get (trying to go in your direction in terms of tone):
still a bit too clean perhaps, but a bit of noise can be added back if needed.
A couple of possible differences between our setups:
-
I use a very neutral profile for denoising, essentially applying only exposure compensation (EDIT: and LMMSE demosaicing)
-
I export in Rec2020
-
I rescale before denoising, because otherwise it takes forever on my modest CPU
Thanks for the tips. I used your approach except that I worked with the full sized image. After denoising, I brought it back into ART for final adjustments, exported, and only then resized. The denoising process is slow, but I think it’s faster than DxO on the VM. This version is definitely better than my first attempt. Still maybe not at DxO’s level, but impressive nonetheless. I’ll have to experiment with less-extreme examples and see how I like it compared to regular noise reduction.
Rescale/resize definitely reduces/averages odd/difficult noise patterns, which makes it easier to denoise afterwards.
I like your result and previous examples.
good to hear. However, note that I wrote “Rec2020”, not “Linear Rec2020”. Rec2020 has a gamma of ~2.2, and that makes a lot of difference. I write this because I noticed that the jpg you attached has a linear rec2020 profile (which really is not suitable for 8-bit files btw…).
HTH
Where do I find Rec2020? Would that be No profile (passthrough mode)?
