can I force tiling?

Software darktable
1

Hi all,

Im so excited. Sorry about posting so much. I have a question about tiling. Im running dark table from the cli and I see warnings like:

Warning: tiling will not be used even if it cant allocate the required amount of memory: xxxMB.

Why is this and is it possible to force darktable to use tiling? Several modules use a lot of memory, which is fine. Im specifically looking at local contrast.

2

Not all modules support tiling, it’s not something that comes out of the box.
However, this text, “tiling will not be used”, does not appear in the source code, anywhere. What version of darktable are you using?

1 Like
3

Whatever the lastest is. I think 5.4.1 release macos. Let me post a sample tonight.

4

The exact text is

IOP_CS_RGB Warning: processed without tiling even if memory requirements are not met

I fpund this in Searching the repo

I dont know what it means other than what is says. I’ve never been very good with tertiary operators in c, especially multiple ones in a single function.

1 Like
5 
(fitting)
       ? ""
       : " Warning: processed without tiling even if memory requirements are not met",

fitting means the processing fits in the available memory. It’s set to:

  const size_t available = dt_get_available_pipe_mem(piece->pipe);
  const size_t total = factor * width * height * bpp + overhead;

  return total <= available;

_piece_may_tile checks whether tiling is applicable (this code is something about masks, as I understand – IOP_FLAGS_WRITE_DETAILS means the module ‘provides the scharr mask used by details’).

static inline gboolean _piece_may_tile(const dt_dev_pixelpipe_iop_t *piece)
{
  return piece->process_tiling_ready
        && !(piece->pipe->want_detail_mask
             && piece->module->flags() & IOP_FLAGS_WRITE_DETAILS);
}

That process_tiling_ready is set to true if(module->flags() & IOP_FLAGS_ALLOW_TILING) (the module provides support), but some modules, when running certain algorithms, may disable it (when not all code paths in the module support tiling).

So tiling can only be enabled if the module supports it (for the current operation), and the detail mask generation does not block it.

But anyway, the answer is no, you cannot force tiling.

2 Likes
6

Also, it’s a warning (only), which means it can be ignored unless you see problems in the image. If you do see issues, the warnings help you in locating the cause.

Did you get GPU processing for that module? GPU memory is smaller than main memory, so an image may not fit in GPU memory without tilling. In that case, dt falls back to CPU processing, which uses the larger main memory (but normally is a lot slower). The value given in the warning message may give a hint in this direction…

Lack of GPU memory is also one of the reasons “weaker” GPU’s may not accelerate processing: the speed gain from the parallel processing is lost in the overhead needed for tiling (or just simple memory transfers).

2 Likes
7

No but I did get sigmoid to work. Local contrast for my massive image needed something like 250gb of ram and segfaulted. I couldn’t find a good reference on what modules could do tiling but it makes sense that some can split the image up and others cant.

AI hinted that local contrast may use tiling but I dont trust it and I couldn’t find any darktable documents.

8

Hmm, how do I set the factor? Yes, it may be slow but im not in a huge rush. I also got very strange results running this on the gpu. The CPU seems far more consistent.

9

It’s not documented, because end users normally shouldn’t be concerned with this. Local contrast can use tiling in bilateral grid mode, but not in local laplacian.

You can see tiling in the logs, if you enable -d tiling.

10

Perfect. Thanks. Ill try that. I found the source code for the bilateral module and it looked like tiling was available but I understand some of this stuff cant work on tiles.

Thanks for the heads up.