That’s a beginning, we are on the same page 
I ever hoped, there is one or some devs, which are in timelapse as well and would support it, or those devs who are the main acts for Eposure, Whitebalance and xmp-definition would accept to go step by step and we try…
It seems to me like the functionality here would be better implemented as a separate tool…
A Python script could interpolate the desired parameters between keyframes and generate outputs to potentially command any RAW processor. Additionally, analysis could be done on frame content to determine parameters for the next frame.
A while ago there was a Python script that implemented key aspects of LRTimelapse functionality (such as dynamically adjusting camera exposure settings during the capture process by analyzing each image and performing “filtered” exposure adjustment similar to what Sony’s in-camera timelapse tool does), I’ll see if I can find it when I get back.
Hello guys,
let me try again an approach and keep being a PITA
Today we find something like this in dt’s xmp:
darktable:operation="temperature"
darktable:enabled="1"
darktable:modversion="3"
darktable:params="046c32400000803f8c4dd63f0000807f"
It is understandable for me, that behind “darktable:params” is all the power of dt, like:
So indeed I would also be very very careful, if I wanna touch a running system.
But just one question:
Is it imaginable, to extract the values, which are interesting for TL and have just those two (temp and tint) available in an editable, additional field? The rest in params should be untouched, just not representing temp/tint anymore.
Here is an simplified example, what I mean:
darktable:operation="temperature"
darktable:enabled="1"
darktable:modversion="3"
darktable:params="046c32400000803f8c4dd63f0000807f" (example !)
darktable:temperature="5748"
darktable:tint="1.012"
Something like this for exposure likewise…
To avoid we damage something, which just runs well today, I propose:
Like this, someone could change this values from outside, theoretically with an simple editor (which at the end is, what LRT does when it exports its work).
Practically I have manually edited WB-temp with an RT .pp3 file tonight and it worked (after changing the temp and open the pic with RT again, RT interpreted that new value correctly and calculated the effects to RGB automatically).
Yes, definitely, that’s actually done in darktable already (when we read those params). I even think that if you didn’t enable the XMP compression in preferences, they are already stored in clear.
Reading your whole post, I wonder actually how much work it would be to just re-implement the feature from scratch in dt. Matching white balance and average brightness is not rocket science.
I just tried with switched off compression, the values are still not really stored in clear but (assumed) bases64
Thanks for your supportive attitude.
I do admit, backwards compatibility could be a pain. Another good reason, to pack such things in fresh modules only, to not hurt, what we have already.
There was some communication on github about how to decode the xmps, maybe have a look there (don’t remember which issue or pr). I still don’t think special modules are needed to interface to lrtimelapse, it’s all there.
I didn’t read the whole thread, but I have a more or less working open source timelapse workflow. Darktable’s exposure module does already do a great job for deflickering and equalizing the exposure. Then it’s just a question of tuning filmic (filmicRGB now). A good starting point is the picture with highest dynamic range of the whole sequence. Then edit as usual (I like local contrast & tone curves) and copy the history stack. Be careful with masks, however, parametric masks can be helpful to keep the highlights in check. The whole process is a bit more involved and I mean to write an Howto, essentially documenting my whole workflow, but I’m drowning in work atm.
The final piece (actually this should be done before all of the editing) is smoothing out the white balance if you shoot your timelapse in AWB (*), like I do. I’ve written a Python script which reads the WB coefficients from the RAW files, applies a smoothing filter (google Savitzky-Golay) and writes it to the XMP files. The params in the XMP file are HEX-packed binary structs, for WB that’s only 4 doubles. Pretty easy to interface with Python once you’ve figured it out. However, my script is CLI only and not very polished but I’m satisfied with the results.
I’ve recently extended the script to allow interpolation (not smoothing!) of exposure and graduatend based on keyframes. That means, pick some files of your sequence, tune exposure (very rarely necessary) and / or graduatend and automatically interpolate all files in between. I still to hacky for a release though.
Here are some Videos I processed roughly following my workflow & using the script:
Nice, I especially the Burg Rabeneck movie. How many images per minute do you usually shoot?
Thank you! 
Depends on the subject. Astro ususally with 30s intervals, so I can expose long enough when it’s dark. But it’s also subject to the 500 rule for astro shots 
For clouds I find ~4s good, depending if I want a sunrise or sunset transition that needs to be longer.
Day-to-Night and Night-to-Day is difficult, hence the term “holy grail”. At the moment I’m experimenting with fixed 4s intervals (Shutter speed mode and a variable ND filter) and 2s pauses. Deflickering can be done in SW, so I don’t care about my aperture changing. The upside is that I can use f/16 for a nice sunstar and once the sun has set the aperture will open automatically. The downside is (potential) flickering and that it’s getting to dark for the VND later at night, so you’d have to remove it, which means touching the camera, which is dangerous 
Btw I also found that you want to be careful about the lens correction module. Different apertures mean different vignetting correction. I now only do distortion & TCA, which seems to work fine.
Looks really nice!
Would you mind to put you python script on GitHub, even it’s «too hacky for a release»? It could be a useful starting point for me anyway, I’m looking for something since a while and didn’t have an idea where to start.
I’m looking forward to this!
Here you go
Thank you very much!
I’d really love to see this integrated into dt. @anon41087856 any change you will have a look into doing this, or would it be faster if I would start learning c now?
Not going to discourage you, but darktable is a vast C++ code base which is more or less a framework around modules. Even if you’re going to learn C++ now it’ll take you some time until you understand darktable’s code base which is (afaik) pretty undocumented. In addition, this is not a standard module in the sense of take pixels from the pipeline, modify pixels and write them back, but rather read information from a group of images and apply it to a group of images, a process darktable was never designed for. So prepare for some hacky solutions here. I would’ve loved to this in LUA though but the API isn’t there, that’s why I “reverse-engineered” the XMP interface.
Apart from that, god speed to you!
I’ve seen the script and all it does is calculating and applying smoothed out WB across set of images (sorry for trivializing actually great piece of work). Technically it can be either an operation in DT in “selected files” module (so you’d have to learn C) or LUA script that would do the same So you can either learn C or LUA. I think lua script can also call Python script that @jchnkl done (lua scripts can call external binaries to do processing). So ultimatelly options are - introduce this as an op in C, introduce it as an op using LUA (total rewrite in lua) or intruduce it as an op in lua dependent on python script.
My stack of tasks is pretty much full for at least the 2 next months, so if you are able to learn C in less than 2-3 months, chances are you will be faster than I am.
Well, you could do a “module” for the lighttable (src/libs/copy_history.c is close to what you do here), code a SQL request to fetch exposures/WB for the selected list of pictures, unroll your interpolation of parameters, save the SQL in database and refresh XMP files.
Great idea and doesn’t sound to hard either.
I’ll be gone till mid-march soon, but I’ll keep that in mind! Thanks for the hint!
Now that darktable has arbitrary module ordering, making a toolchain of module invocations in lua would be a powerful way to process multiple images.
I recently completed a batch capability for my own software, and it has surpassed my expectations for improving the efficiency of my workflow. For instance, after I’ve produced my proof JPEGs after a day’s shooting, if I find something like a white balance correction that applies to a series of images, I can open one of them (which opens the raw file and re-applies the toolchain to make the JPEG), change the white balance, then I delete the other proofs i want to apply the change to and kick of the batch dialog, which will process new JPEGs with the modified processing. It’s not lua, but the concept is the same: have a scripting language that exposes the same image processing API as the GUI program uses.
Food for thought…