Timelapse photography and darktable

[David_LaCivita]

@jchnkl

I mean to write an Howto

I’m looking forward to this!

[jchnkl (Jochen Keil)]

Here you go

Fixing White Balance (Color) Flicker in Timelapse Videos with a Python Script for darktable

Hello, over the last one or two years a gathered some experience with Open Source Timelapse Post-Processing. While I don’t have the time to write a How-To (yet) I want to share a Python script which I wrote to significantly improve the results. Note, that the script works in combination with darktable, so if you don’t want to or can’t use darktable you can stop reading now. First, a bit of background information of why I wrote this script and how it works. Generally, if you look…

[martin_sch (Martin Schilliger)]

Thank you very much!

[Tobias]

I’d really love to see this integrated into dt. @anon41087856 any change you will have a look into doing this, or would it be faster if I would start learning c now?

[jchnkl (Jochen Keil)]

Not going to discourage you, but darktable is a vast C++ code base which is more or less a framework around modules. Even if you’re going to learn C++ now it’ll take you some time until you understand darktable’s code base which is (afaik) pretty undocumented. In addition, this is not a standard module in the sense of take pixels from the pipeline, modify pixels and write them back, but rather read information from a group of images and apply it to a group of images, a process darktable was never designed for. So prepare for some hacky solutions here. I would’ve loved to this in LUA though but the API isn’t there, that’s why I “reverse-engineered” the XMP interface.

Apart from that, god speed to you!

[johnny-bit (Hubert Kowalski)]

I’ve seen the script and all it does is calculating and applying smoothed out WB across set of images (sorry for trivializing actually great piece of work). Technically it can be either an operation in DT in “selected files” module (so you’d have to learn C) or LUA script that would do the same So you can either learn C or LUA. I think lua script can also call Python script that @jchnkl done (lua scripts can call external binaries to do processing). So ultimatelly options are - introduce this as an op in C, introduce it as an op using LUA (total rewrite in lua) or intruduce it as an op in lua dependent on python script.

[anon41087856]

My stack of tasks is pretty much full for at least the 2 next months, so if you are able to learn C in less than 2-3 months, chances are you will be faster than I am.

[anon41087856]

@jchnkl

In addition, this is not a standard module in the sense of take pixels from the pipeline, modify pixels and write them back, but rather read information from a group of images and apply it to a group of images, a process darktable was never designed for. So prepare for some hacky solutions here.

Well, you could do a “module” for the lighttable (src/libs/copy_history.c is close to what you do here), code a SQL request to fetch exposures/WB for the selected list of pictures, unroll your interpolation of parameters, save the SQL in database and refresh XMP files.

[jchnkl (Jochen Keil)]

Great idea and doesn’t sound to hard either.
I’ll be gone till mid-march soon, but I’ll keep that in mind! Thanks for the hint!

[ggbutcher (Glenn Butcher)]

Now that darktable has arbitrary module ordering, making a toolchain of module invocations in lua would be a powerful way to process multiple images.

I recently completed a batch capability for my own software, and it has surpassed my expectations for improving the efficiency of my workflow. For instance, after I’ve produced my proof JPEGs after a day’s shooting, if I find something like a white balance correction that applies to a series of images, I can open one of them (which opens the raw file and re-applies the toolchain to make the JPEG), change the white balance, then I delete the other proofs i want to apply the change to and kick of the batch dialog, which will process new JPEGs with the modified processing. It’s not lua, but the concept is the same: have a scripting language that exposes the same image processing API as the GUI program uses.

Food for thought…

[anon41087856]

@ggbutcher

Now that darktable has arbitrary module ordering, making a toolchain of module invocations in lua would be a powerful way to process multiple images.

No, the modules that you can reorder are the pixel pipe ones (where “module” means a GUI widget and an image operation filter, located in src/iop). Those are not accessible from Lua. Lua let you play only with pipeline I/O and the lighttable modules (where “module” means only a GUI widget to do arbitrary stuff not affecting pixels, those are located in src/libs)

[ggbutcher (Glenn Butcher)]

Okay, some of what I’m getting at is already in darktable-cli. From my initial read of the doc, one needs a previously-saved XMP with the desired history stack. I’m not a large fan of history stacks, I’d rather just have a ‘toolchain stack’ of the parameters I finally settled on, not all the less-than-desirable iterations it took me to get there. Okay, further reading reveals “compress history stack”, which gets rid of all but the essential information needed to process the image with the final editing decisions. That gets me there…

I still prefer scripting my toolchains…

[Tobias]

Just to be complete her, I found this video “Timelapse stabilised with Hugin tools”:

Timelapse stabilised with Hugin tools

And here is the code:
https://bitbucket.org/brunopostle/panotools-script/src/eee8f84f40fda3d6d1d5954fa7bb20848b8041fc/bin/nona-deshake?at=default&fileviewer=file-view-default

[Tobias]

@anon41087856

…, so if you are able to learn C in less than 2-3 months, chances are you will be faster than I am.

I think you will be faster.

@anon41087856

Well, you could do a “module” for the lighttable ( src/libs/copy_history.c is close to what you do here), code a SQL request to fetch exposures/WB for the selected list of pictures, unroll your interpolation of parameters, save the SQL in database and refresh XMP files.

This sounds like a very clear plan and a good starting point.

[AxelG]

Ladies and Gentlemen,

I appreciate the conversation here, which I followed silently.

Also I kept back off for the purpose of letting dt-3.0.0 come to earth and getting calm.

Now I would like to go back a little to my initial intention. Hence I have just now opened a Feature Request on github and seriously hope for any helpful support…

https://github.com/darktable-org/darktable/issues/4250

Sincerely
Axel

[AxelG]

With the upcoming darktable 3.2, we will get the new introspection mode , which @dterrahe has been implemented. I am very very grateful for this work and happy that I have had a little minor role of being part of the testing team.

It think there is still a long way to go and patience is needed

@jchnkl meanwhile has realeased dtlapse, which I still need to try, but looks very promising.

• Some of you may want to merely use FOSS, then dtlapse probably is the long awaited diamond
• those looking forward to the very mature LRTimelapse, dtlapse might be the tool, makes the required patience a bit easier to accept.

[Tobias]

Could you explain what the benefits of the new introspection mode for timelaps are?

[AxelG]

AFAIK this is a pre-requisite, that modules can “talk” to each other or control other modules and also the base for human legible PARAMS.

But I am not a coder, so easily I explained it very wrong

[Ayush_Gupta]

Hi @AxelG I know you had been pursuing this idea very passionately last year.
Do you know if there is any new development on this or has this lost steam?

[AxelG]

Hi,

after introspection, unfortunately nothing else happened, besides me or Hubert pushing the git issue away from auto-closure.

At least me would like to keep the steam up, however, I’m still 100%+ busy with house renovation, hence I cannot push it atm. This scenario might last until next summer, I’m afraid. :-/