Introducing neural restore module – raw denoise, denoise, and upscale

KarlMagnusLarsson · May 3, 2026, 10:23am

This is what I get if I do neural restore -> raw denoise, strength = 100% on A7407127.ARW on Linux Debian 13.4, NVIDIA CUDA (EDIT: When I succesfully create a denosed dng-file and run -d ai).

EDIT2: I was on a PR, sorry for that. This is using latest git master:

darktable 5.5.0+1124~gb8a2ee3529
Copyright (C) 2012-2026 Johannes Hanika and other contributors.

Compile options:
  Bit depth              -> 64 bit
  Exiv2                  -> 0.28.5
  Lensfun                -> 0.3.4
  Debug                  -> DISABLED
  SSE2 optimizations     -> ENABLED
  OpenMP                 -> ENABLED
  OpenCL                 -> ENABLED
  Lua                    -> ENABLED  - API version 9.7.0
  Colord                 -> ENABLED
  gPhoto2                -> ENABLED  - Camera tethering is available
  OSMGpsMap              -> ENABLED  - Map view is available
  GMIC                   -> DISABLED - Compressed LUTs are NOT supported
  GraphicsMagick         -> ENABLED
  ImageMagick            -> DISABLED
  libavif                -> ENABLED
  libheif                -> ENABLED
  libjxl                 -> ENABLED
  LibRaw                 -> ENABLED  - Version 0.22.0-Release
  OpenJPEG               -> ENABLED
  OpenEXR                -> ENABLED
  WebP                   -> ENABLED
  AI                     -> ENABLED

See https://www.darktable.org/resources/ for detailed documentation.
See https://github.com/darktable-org/darktable/issues/new/choose to report bugs.

[dt starting] as : ./install/bin/darktable --configdir /home/magnus/test/darktable-test/config --cachedir /home/magnus/test/darktable-test/cache -d ai
     0,0585 [ai_models] initialized: models_dir=/home/magnus/.local/share/darktable/models, cache_dir=/home/magnus/test/darktable-test/cache/ai_downloads
     0,0585 [ai_models] using repository: darktable-org/darktable-ai
     0,0586 [ai_models] registered model: mask sam2.1 hiera small (mask-object-sam21-small)
     0,0586 [ai_models] registered model: mask segnext vitb-sax2 hq (mask-object-segnext-b2hq)
     0,0586 [ai_models] registered model: denoise nind (denoise-nind)
     0,0586 [ai_models] registered model: denoise nafnet small (denoise-nafnet)
     0,0586 [ai_models] registered model: raw denoise nind (rawdenoise-nind)
     0,0586 [ai_models] registered model: upscale bsrgan (upscale-bsrgan)
     0,0586 [ai_models] registry loaded: 6 models from /home/magnus/test/darktable-test/install/share/darktable/ai_models.json
     1.3762 [darktable_ai] dt_ai_env_init start.
     1.3763 [darktable_ai] discovered: denoise nafnet small (denoise-nafnet, backend=onnx)
     1.3763 [darktable_ai] discovered: raw denoise nind (rawdenoise-nind, backend=onnx)
     1.3764 [darktable_ai] discovered: mask sam2.1 hiera small (mask-object-sam21-small, backend=onnx)
     1.3764 [darktable_ai] discovered: upscale bsrgan (upscale-bsrgan, backend=onnx)
     1.3764 [darktable_ai] discovered: mask segnext vitb-sax2 hq (mask-object-segnext-b2hq, backend=onnx)
     1.3764 [darktable_ai] discovered: denoise nind (denoise-nind, backend=onnx)
sh: 1: cd: can't cd to /home/magnus/test/darktable-test/config/lua
fatal: not a git repository (or any of the parent directories): .git
sh: 1: cd: can't cd to /home/magnus/test/darktable-test/config/lua
fatal: not a git repository (or any of the parent directories): .git
find: ‘/home/magnus/test/darktable-test/config/lua’: No such file or directory
     4.1254 [restore] variant 'bayer': file=model_bayer.onnx input_kind=bayer_v1
     4.1254 [restore] model rawdenoise-nind: coreml_cpu_only=true (ep_flags=1)
     4.1261 [darktable_ai] loaded ORT 1.24.4 from '/home/magnus/.local/lib/onnxruntime-cuda/libonnxruntime.so'
     4.1261 [darktable_ai] execution provider: CUDA
     4.1348 [darktable_ai] loading: /home/magnus/.local/share/darktable/models/rawdenoise-nind/model_bayer.onnx
     4.1349 [darktable_ai] attempting to enable NVIDIA CUDA...
     4.1380 [darktable_ai] CUDA driver 12.4, runtime 12.4
     4.1843 [darktable_ai] NVIDIA CUDA enabled successfully.
     4.2643 [neural_restore] job started: task=raw denoise, scale=1, images=1
     4.2644 [neural_restore] processing imgid 1 -> /home/magnus/test/raw_others/A7407127_raw-denoise_1.dng
     4.3305 [neural_restore] imgid 1: flags=0x10640 channels=1 filters=0x94949494 (bayer)
     4.3584 [restore_raw_bayer] 7168x5120 sensor (CFA origin 0,0), working 3584x2560 packed, tile T=1024, 4x3 grid (12 tiles)
     4.3620 [restore_raw_bayer] raw CFA range [0.0, 16383.0], black=[512,512,512,512] white=15360 wb_coeffs=[2229.000,1024.000,1965.000,0.000] wb_norm=[1.000,1.000,1.000]
     4.3755 [restore_raw_bayer] tile0 model_input range R=[-0.016,0.029] G1=[-0.017,0.072] G2=[-0.015,0.065] B=[-0.015,0.061]
     4.7547 [restore_raw_bayer] tile0 model_output range R=[0.000,0.021] G=[0.001,0.058] B=[0.001,0.051] in_mean=0.012 out_mean=-10430834.238 gain=-1.186e-09
     7.8004 [restore_raw_bayer] cfa_out u16 range [0, 15360] mean=878 (DNG will advertise black~512 white=15360)
     7.8090 [neural_restore] embedded JPEG preview from source 6224x4672 (3273098 bytes)
     7.9010 [neural_restore] imported imgid=3: /home/magnus/test/raw_others/A7407127_raw-denoise_1.dng

anry · May 3, 2026, 10:37am

Can you delete and download again raw denoise model please? And check if processing time changes.

anry · May 3, 2026, 10:55am

The issue Entry 0x828d not found was already fixed in master. Please, pull the master or use latest nightly build. And let me know if this helps.

anry · May 3, 2026, 11:15am

Thank you for sharing files, it helped a lot. Your DNG was missing the CFA pattern tags because some libtiff builds drop the field info between IFD0 and SubIFD0. Now fixed in PR #20913.

Thad_E_Ginathom · May 3, 2026, 3:01pm

Great! But it’s been a bad day: my last attempt completely flattened my system, which was unusable for about half an hour, and I still couldn’t load the dng.

I’ll revisit all this tomorrow, build the next nightly and let you know.

Maybe I’ll take a couple of pics in the dark tonight, and try to get some actual noise to play with . It is not an urgent need for me, personally, but still, it’s a great tool to have, and if I can help find some problems it’s well worth contributing testing time

dtrtuser · May 3, 2026, 4:15pm

Hi Andrii,

Deleted and reloaded the model using AI preferences screen.
Restarted dt, tested with Fuji: works fine and very fast on GPU (couple of seconds).
Tried on non Fuji raw files: took about 10 seconds but all preview images came out all black when strength of denoise set to 100%.
Restarted dt with -d ai without changing anything else and Olympus image was processed properly but taking same very long time for preview (slightly more than 1 minute).

anry · May 3, 2026, 5:12pm

@dtrtuser, Bayer raw denoise model turns out to be a little bit too complex for efficient graph construction on some GPUs. I don’t have immediate solution to this. Most probably it will require changes in the model architecture. I will discuss this issue with the author of the model Benoit Brummer.

dtrtuser · May 3, 2026, 8:05pm

Thanks Andrii.

Just in case it helps, here is the log when the image comes back all black:

   196.2220 [darktable_ai] loading: C:\Users\dtrtuser\AppData\Local\darktable\models\rawdenoise-nind\model_bayer.onnx
   196.2220 [darktable_ai] attempting to enable Windows DirectML...
   196.3831 [darktable_ai] Windows DirectML enabled successfully.
   196.7461 [restore_raw_bayer] preview failed, retrying at T=1536
   206.6319 [neural_restore] raw preview: inference returned err=0 src=000002033ab7e040 denoised=000002033a0bc040 requested=560x574 actual=560x574
   209.1226 [neural_restore] preview: reusing 5240x3912, scale=1, export_size=0
   212.3925 [darktable_ai] loading: C:\Users\dtrtuser\AppData\Local\darktable\models\denoise-nind\model.onnx
   212.3925 [darktable_ai] attempting to enable Windows DirectML...
   212.5527 [darktable_ai] Windows DirectML enabled successfully.
   212.9370 [neural_restore] preview: tiled inference 512x512
   212.9373 [restore_rgb] working profile color matrices ready
   212.9373 [restore_rgb] tiling 512x512 (scale=1) -> 512x512, 1x1 grid (1 tiles, T=1536)
   215.0714 [neural_restore] preview cache hit for raw denoise
   217.6587 [neural_restore] raw preview: imgid=328 bayer patch=(0.446,0.451) widget=560x575 filters=0x94949494
   217.7457 [restore] variant 'bayer': file=model_bayer.onnx input_kind=bayer_v1
   217.7457 [restore] tile size 2048 (scale=1, need 4896MB, budget 11180MB)
   217.7457 [restore] model rawdenoise-nind: coreml_cpu_only=true (ep_flags=1)
   217.7458 [darktable_ai] loading: C:\Users\dtrtuser\AppData\Local\darktable\models\rawdenoise-nind\model_bayer.onnx
   217.7458 [darktable_ai] attempting to enable Windows DirectML...
   217.8998 [darktable_ai] Windows DirectML enabled successfully.
   218.2637 [neural_restore] raw preview: full=5240x3912 ori=0x0 patch_center=(0.446,0.451) -> sensor=(2058,1476 560x574) bayer
   222.3554 [neural_restore] raw preview: inference returned err=0 src=000002033aeef040 denoised=000002032c8f5040 requested=560x574 actual=560x574
   255.8253 [neural_restore] raw preview: imgid=328 bayer patch=(0.062,0.480) widget=560x575 filters=0x94949494
   255.8253 [neural_restore] raw preview: full=5240x3912 ori=0x0 patch_center=(0.062,0.480) -> sensor=(46,1588 560x574) bayer
e [ 1 ; 3 1 m 2 0 2 6 - 0 5 - 0 3   1 2 : 5 8 : 3 9 . 5 2 7 3 7 9 6   [ E : o n n x r u n t i m e : ,   s e q u e n t i a l _ e x e c u t o r . c c : 5 7 2   o n n x r u n t i m e : : E x e c u t e K e r n e l ]   N o n - z e r o   s t a t u s   c o d e   r e t u r n e d   w h i l e   r u n n i n g   D m l F u s e d N o d e _ 0 _ 0   n o d e .   N a m e : ' D m l F u s e d N o d e _ 0 _ 0 '   S t a t u s   M e s s a g e :   E : \ _ w o r k \ 1 \ s \ o n n x r u n t i m e \ c o r e \ p r o v i d e r s \ d m l \ D m l E x e c u t i o n P r o v i d e r \ s r c \ D m l G r a p h F u s i o n H e l p e r . c p p ( 1 0 7 8 ) \ o n n x r u n t i m e . d l l ! 0 0 0 0 7 F F F 3 1 0 C 9 1 C D :   ( c a l l e r :   0 0 0 0 7 F F F 3 1 1 5 5 7 E 3 )   E x c e p t i o n ( 4 )   t i d ( 5 1 2 4 )   8 8 7 A 0 0 0 6   T h e   G P U   w i l l   n o t   r e s p o n d   t o   m o r e   c o m m a n d s ,   m o s t   l i k e l y   b e c a u s e   o f   a n   i n v a l i d   c o m m a n d   p a s s e d   b y   t h e   c a l l i n g   a p p l i c a t i o n . 
 
 e [ m 
    255.8883 [darktable_ai] run error: Non-zero status code returned while running DmlFusedNode_0_0 node. Name:'DmlFusedNode_0_0' Status Message: E:\_work\1\s\onnxruntime\core\providers\dml\DmlExecutionProvider\src\DmlGraphFusionHelper.cpp(1078)\onnxruntime.dll!00007FFF310C91CD: (caller: 00007FFF311557E3) Exception(4) tid(5124) 887A0006 The GPU will not respond to more commands, most likely because of an invalid command passed by the calling application.

   259.6956 [darktable_ai] loading: C:\Users\dtrtuser\AppData\Local\darktable\models\rawdenoise-nind\model_bayer.onnx
   259.6957 [darktable_ai] attempting to enable Windows DirectML...
   259.8592 [darktable_ai] Windows DirectML enabled successfully.
   260.2208 [restore_raw_bayer] preview failed, retrying at T=1536
   269.9103 [neural_restore] raw preview: inference returned err=0 src=000002032895e040 denoised=000002032859a040 requested=560x574 actual=560x574

anry · May 3, 2026, 8:06pm

I opened a discussion in GitHub Issue on possible ways to improve user experience when using neural restore module.

github.com/darktable-org/darktable

Discussion: neural restore module architecture

opened 07:58PM - 03 May 26 UTC

andriiryzhkov

Hi all, I've been reading the comments coming in from early adopters and fellow… developers – here on GitHub and over on the [pixls.us darktable forum](https://discuss.pixls.us/tag/darktable) – and I want to open this up for a wider conversation before I commit to one direction. neural restore is still young and now is the time to shape it. ## what I'm hearing A couple of things keep coming up: - The split before/after **preview is too small to really judge the result**. People end up moving the patch around several times just to decide on a strength. - Terry Pinfold (on pixls.us) shared that he runs neural restore and then uses the **composition** module afterwards to blend the source and the denoised file – essentially building his own strength control because the one in the module isn't enough on its own. Both point at the same thing: people want to see the effect on the **full image**, and they want fine control over how strong it is. ## three directions I'm considering ### 1. keep it as is – utility module, lighttable + darkroom The current setup. Strength is a slider in the module's panel. - ✅ batch processing across selected images in lighttable - ❌ no full-image preview – the module can't reuse pipe data to render in darkroom - ❌ small patch preview is the only way to judge ### 2. utility module, darkroom-only Drop lighttable. Output is still a separate file (DNG / TIFF). - ✅ the strength slider could apply directly to the **full main image** in darkroom, post-inference – pretty small change from where we are - ✅ no more guesswork on a tiny patch - ❌ no more batch processing ### 3. processing module in darkroom (raw denoise only) Make raw denoise a real iop in the darkroom pipeline. No separate output file visible to the user. Important caveat: I think this is **only realistic for raw denoise**. Its input is sensor data, which doesn't change as you edit – the inference result stays valid. RGB denoise's input depends on every iop above it, so any tweak invalidates the cache. So path 3 effectively splits the module: raw as iop, denoise and upscale stay utility-style. - ✅ darkroom-native – strength, blending, masks all "just work" through the pipe - ✅ same plumbing helps future stable-input AI features (depth, embeddings, …) - ❌ AI inference is far slower than a normal pipe step – can't re-run on every history change, so we need to cache the result somewhere Two options for the cache: **reuse darktable's existing cache** (`~/.cache/darktable/`) – simpler, but doesn't travel with the image, so library migrations / backups / multi-machine sync mean redoing inference. Or a **new sidecar next to the source** (e.g. `image.dtdata`) – portable, but a new on-disk format to define and maintain. This is the biggest refactor of the three, and the most future-proof for stable-input AI features. ## about timing To be clear: I'm not proposing to do all of this for the next release. What I'd like out of this discussion is a **north star** – the direction we agree neural restore should grow towards. From there, what's actually feasible per release: - Path 2 is a relatively small change from where we are today and could realistically land in the **next release**. - Path 3 is a much bigger refactor (new cache format or cache-infrastructure changes, iop integration, model-input contract for stable AI features, …) and is **not feasible for the next release** – it's a multi-cycle effort. So we can move incrementally even if the long-term destination is path 3. ## what I'd love to hear - Which direction fits your workflow best as the long-term destination? - For path 3 – which caching approach feels right: reuse the existing darktable cache (simpler, doesn't travel with the image) or a new `.dtdata`-style sidecar (portable, but new format)? And is splitting the module (raw as iop, RGB / upscale as utility) acceptable, or does it feel inconsistent? - Am I missing something? A fourth option, an existing darktable pattern I should reuse, edge cases I haven't thought of? All input welcome – thanks!

I will not duplicate full text here, just highlight that I see three options of how this module can be frames from the user perspective. Core features stays the same.

I welcome you to read issue, provide your thoughts on GitHub or directly in this topic – whichever is more convenient for you.

dtrtuser · May 4, 2026, 3:19am

Hi Andrii,

I thought I could help with the performance issue by using Windsurf (with SWE-1.6 and Claude Sonnet 4.6). Initial tests were promissing (in less than a second for preview and a couple of seconds for processing the whole image) but unfortunately it introduced new bugs (crash when processing Nikon D300 images and banding on Sony A7R5)

After few hours trying to debug, the AI gave up with this:

Note on performance: T=2048 is the only valid option for this model, so the speed will be the original (pre-optimization) rate. The performance improvement requires either a model retrained for smaller inputs, or a different approach unrelated to tile size.

Also, after few more hours of effort, got this (might as well be pure garbage):

Three-Way Comparison

Backend	Tile time	Total	Output	Issues
CPU	16.5s	35s	Correct	None
CUDA	41.4s	114s	Correct	VRAM overflow → PCIe bottleneck
DirectML	4.3s→63s	175s	Zeros, then correct	FP16 failure → GPU hang → TDR

Conclusion

CPU wins on this hardware because it accesses system RAM directly — no VRAM limit, no PCIe overhead. GPU would only outperform CPU if the entire model’s intermediate activations fit within VRAM. For this model at T=2048, that requires significantly more than 8GB — likely 16-24GB VRAM (RTX 4090 territory).

The coreml_cpu_only=true flag in the model manifest exists precisely for this reason. The model authors should add an equivalent directml_cpu_only=true (or a generic gpu_cpu_only=true ) flag to automatically route CUDA/DirectML users to CPU too.

anry · May 4, 2026, 2:05pm

directml_cpu_only=true could help, but it is only cosmetics.

Thad_E_Ginathom · May 4, 2026, 2:49pm

I have got the NVIDIA stuff installed and apparently working.

The neural denoise (middle tab) seems to be working nicely. I photographed some flowers in the evening garden at 25600. It has done a nice job.

Curious feature: there are some red speckles, like hot pixls, but they are not there in the final output.

Raw denoise… I’m going from bad to worse. I have rebuilt from the current nightly today; I have deleted and redownloaded all the models. Even in 20 minutes I did not get any dng, not even a created file.

As soon as I press process, my desktop pretty-much freezes.

I might leave one running when I go to bed tonight, to see if it does eventually produce a dng and if it is usable. And do it with debug.

Otherwise, I’ll leave that be for a while. Others have it running: I’m sure it must be me!

Not a big problem: profiled denoise is almost always enough for my pics, and I can see neural denoise will be a useful emergency measure.

anry · May 4, 2026, 2:53pm

Can you share more details about your septup? OS, GPU, what ONNX Runtime you installed (if any) additionally.

-d ai logs may also help.

Thad_E_Ginathom · May 4, 2026, 3:03pm

Sure! But give me a one or two non-nerd days first

afre · May 4, 2026, 9:41pm

@Thad_E_Ginathom A more elegant solution than spoiler is to use details. For example:

[details=stdout]
[/details]

stdout

priort · May 5, 2026, 3:39am

The routine seems to complete when I do the raw denoise on an image but these lines follow the DNG import… Win 11 CUDA - ORT

(org.darktable.darktable:12124): GLib-GObject-CRITICAL **: 23:07:17.650: g_object_set_data: assertion ‘G_IS_OBJECT (object)’ failed

(org.darktable.darktable:12124): Gtk-CRITICAL **: 23:07:17.650: gtk_tree_model_foreach: assertion ‘GTK_IS_TREE_MODEL (model)’ failed

(org.darktable.darktable:12124): GLib-GObject-CRITICAL **: 23:07:27.138: g_object_set_data: assertion ‘G_IS_OBJECT (object)’ failed

(org.darktable.darktable:12124): Gtk-CRITICAL **: 23:07:27.138: gtk_tree_model_foreach: assertion ‘GTK_IS_TREE_MODEL (model)’ failed

Not sure if this means anything or its more of a warning type comment…

anry · May 5, 2026, 10:29am

Is it reproducible or it was a one-time thing? I can not reproduce this error on Windows so far. Maybe you can describe more details about the situation it happens?

priort · May 5, 2026, 12:10pm

I’ll spend a little more time with it tonight and see if I can systematically reproduce it…Thanks for explaining the masking as well…

Thad_E_Ginathom · May 5, 2026, 12:23pm

Thanks: I will. I have used vB-based forums for decades, but only use discourse for this one.

(I was thinking, wouldn’t it be stderr? So I checked. And it isn’t. I know… you’re the programmer and I should have just trusted you )