Annoucement of dtLapse

Oh, don’t worry about it. I’m sure the instructions are clear for most Linux users, unfortunately I am still in the early stages of learning. I am up and running now!
I have a question about capturing the images if that’s OK? During the shoot, do you make adjustments to exposure as the light changes to not loose shadows or highlights?

1 Like

I’ve experimented with different methods.

The simplest method is to use the aperture mode and automatic ISO. That way you’ll set your aperture to a fixed value (E.g. F8) and the camera adjusts the exposure and if the exposure gets too long (on my camera that’s configurable) adjust the ISO. For that method I usually use an EV -1.0 to prevent highlight blowout. Shadows can usually be recovered pretty well.
That’s the (imo) easiest method if your camera supports A-mode and auto-ISO with a maximum exposure time.

A little bit more convoluted is the second approach. Use shutter mode and a variable ND filter with auto ISO. The pitfall here is that variable NDs show a strong “cross” when you turn them to far. Also, most camera’s will just use maximum aperture (e.g. F22 or so) in shutter mode which you also want to avoid because of difraction. I’ve had limited success with this method.

The third approach would be to go fully manual, meaning that you adjust exposure, ISO and aperture manually when required. The backdraw is that you have to touch your camera, every time risking to shake it. You will see even minimal movements later in the footage. My cameras have very tight knobs to turn, so I avoid that method. Also, when it’s cold outside it’s even harder with trembling fingers. And if you’re using a slider or so then you shouldn’t touch your camera at all.

Method number four would be to use a software qdslrdashboard. I haven’t tried it (extensively) yet, but from what I read it looks promising. Backdraw would be higher power consumption which is prohibitive when you are on a multi-day trip without grid access (e.g. backcountry or mountain hike)

Finally you could use a dedicated device like the Timelapse View+ sold by an NZ company. I don’t know much about it, because it sells for around $400.

HTH. Cheers! :slight_smile:

Also entangle or darktable if you’re on Linux.

I’ll have to see how much I can control from the Olympus app but the dials on my camera move easily. It’s also very light and stable on any tripod.

If you’re not doing any kind of movement of the camera, you could probably align the images in hugin and auto crop them before making the video.

Good news everyone!

DtLapse is now on PyPI:

Installation is now as easy as

$ pip install dtlapse

@David_LaCivita It should now install effortlessly :blush:
@darix I’ve also tagged v1.0.0 in gitlab :slight_smile:

4 Likes

thanks for pinging me :slight_smile:

I am definitely flattered about Jochen’s work and will try it out after the current heatwave with my Neowise TL-shots. Exposure (hopefully the automatic mode is working) and WB is already huge improvement, especially for holy grail shots, where night-time-compensation (NTC) is used.

I also appreciate the maturity of LRTimlapse and hence I hope one day dt could reach compatibility level, which we got a huge step closer with the new introspection from @dterrahe, if I understood this correctly. Hence I am still on for my article :smile:

I just did pip install dtlapse and cannot wait, weather getting more normal again :sweat_smile:

1 Like

Wow, that’s amazing!

Are you doing anything with your camera setup to reduce flicker or ramping the exposure from sunset to dusk or does the keyframes and script/algorithm handle all this?

I’m definitely going to have to try this out!!!

Thank you! :blush:

As for my camera setup, it depends. When I’m lazy I just use Aperture (A mode). My Sony does a pretty good job for ramping and rarely requires deflickering. But if it occurs you can use either darktable’s exposure module in automatic mode or the script.

I set White Balance always to AWB, because that way I get an idea of what the White Balance should look like. Especially for Sunrise / Sunset WB changes over time. However, the automatic algorithm seems to be less stable than the light meter. This results in what I call “color flicker”. This can be solved using this script and was the actual reason for writing this script. :slight_smile:

I hope you enjoy using this script and get something out of it!

2 Likes

Ok, dtlapse actually came right on time! I have toyed some with it last week, but now I have a real use case!

I shot a timelapse of the build-up of a tv studio yesterday. So: controlled lighting, easy right? Nope, they assembled a white backdrop, so darktable’s auto-exposure/deflicker came up with a different exposure between the beginning and the end of the video. Keyframing the auto-exposure percentile did the job!

BTW, what are you using to make the final video? Plain ffmpeg or something like Shotcut or Blender?

2 Likes

That’s awesome, I’m so happy to hear that! :blush:

I export my pictures in full resolution to 16bit TIFFs. Then I create a video with ffmpeg:

ffmpeg -framerate 24 -f image2 -i '%04d'.tif -c:v ffv1 output.mkv

Notes:
Check out darktable’s ${SEQUENCE} export variable
ffmpeg does not handle color profiles properly (at least for me) so I set the output color profile to my monitor profile which is probably not the right thing to do but with that I get consistent colors in my videos.
Using FFv1 will generate a huge file, you’ve been warned :slight_smile:
It’s a lossless intra-frame codec, i.e. only compresses the video on a per-frame basis and does not consider the previous or following frames like inter-frame codecs e.g. mpeg. But I haven’t found a lossless inter-frame codec (for linux) yet. Expect the file size to be equal to the size of your TIFF files.

After that I usually generate a smaller video file for editing in Shotcut:

ffmpeg -i output_ffv1_fullres.mkv -c:v libx264 -vf "scale=2560:-1" output_x264_2560x.mkv

I use a symlink for proxy editing:

ln -s output_x264_2560x.mkv output.mkv
shotcut output.mkv
<edit edit edit close shotcut>
rm output.mkv
ln -s output_ffv1_fullres.mkv output.mkv
shotcut output.mkv
<export video with the 4K preset, high quality resampling etc>

There are some filters in Shotcut which work on absolute parameters, but that’s fixable even on a large file. Using the proxy file can give you a good estimate where you want to go especially for placing the soundtracks.

1 Like

very good work! a simple GUI would be nice to have :smiley:

1 Like

This is, what I am struggling with atm… I shoot over night and import in different subfolders acc. to date with RPD. For dt that is not a problem, as I import the next higher folder with all subfolders, but for dtlapse, I donno how to fetch them (maybe I am lacking too much bash know-how)

Indeed I was, but darktable on IRC helped :slight_smile:
dtlapse temperature --xmps ./2020-07-[2][0123456789]/*{9059..9331}*xmp --smooth --plot

Dear @jchnkl,

I am editing with current master of dt (3.3.0+651~gb6bbd0d56) and got the following errors ( I have used that modules):

$ dtlapse bilat --xmps ./2020-07-[2][0123456789]/*{9059…9331}*xmp --smooth --plot

Traceback (most recent call last):
  File "/home/gerber/.local/bin/dtlapse", line 8, in <module>
    sys.exit(main())
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 209, in main
    params = get_params(history, iop.operation)
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 76, in get_params
    raise ValueError
ValueError

$ dtlapse colorbalance --xmps ./2020-07-[2][0123456789]/*{9059…9331}*xmp --smooth --plot

Traceback (most recent call last):
  File "/home/gerber/.local/bin/dtlapse", line 8, in <module>
    sys.exit(main())
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 209, in main
    params = get_params(history, iop.operation)
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 76, in get_params
    raise ValueError
ValueError

$ dtlapse hazeremoval --xmps ./2020-07-[2][0123456789]/*{9059…9331}*xmp --smooth --plot

Traceback (most recent call last):
  File "/home/gerber/.local/bin/dtlapse", line 8, in <module>
    sys.exit(main())
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 209, in main
    params = get_params(history, iop.operation)
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 76, in get_params
    raise ValueError
ValueError

$ dtlapse filmicrgb --xmps ./2020-07-[2][0123456789]/*{9059…9331}*xmp --smooth --plot
Could not find modversion 4 for the filmicrgb module.

for filmic, here is my excerpt of the xmp (with modversion=4"):

 <rdf:li
  darktable:num="7"
  darktable:operation="filmicrgb"
  darktable:enabled="1"
  darktable:modversion="4"
  darktable:params="gz02eJybNXOyIwPDjwO2XMsdGBgcoPiEEwRDwCygGv1lFTYgse+8QSB5x7Nn1tgzMCg4guTPnjljywykGaGYAUozMaCCJ+wMDAAfRBW1"
  darktable:multi_name=""
  darktable:multi_priority="0"
  darktable:blendop_version="9"
  darktable:blendop_params="gz11eJxjYGBgkGAAgRNODGiAEV0AJ2iwh+CRyscOAAdeGQQ="/>

what am I doing wrong

Sorry, if too many noob questions:

what does the following tell me (I get the same for temperature)?

$ dtlapse exposure --xmps ./2020-07-2[12]/*{8540..8870}.NEF.xmp --no-backup --plotTraceback (most recent call last):
  File "/home/gerber/.local/bin/dtlapse", line 8, in <module>
sys.exit(main())
  File "/home/gerber/.local/lib/python3.7/site-packages/dtlapse/dtlapse.py", line 236, in main
kind=args.interp_method)
  File "/home/gerber/.local/lib/python3.7/site-packages/scipy/interpolate/interpolate.py", line 529, in __init__
check_finite=False)
  File "/home/gerber/.local/lib/python3.7/site-packages/scipy/interpolate/_bsplines.py", line 817, in make_interp_spline
"match: expected %s, got %s+%s" % (nt-n, nleft, nright))
ValueError: The number of derivatives at boundaries does not match: expected 1, got 0+0

Hi @AxelG,

basically nothing, but didn’t get around to update dtlapse’s modules to the latest modules for darktable (I did not even update my copy of darktable yet…). So for now, all the modules are on 3.0.2.

That probably means that you’re not using enough keyframes and / or didn’t not include the first and the last frame as keyframes.

For interpolation with anything else than “linear” to work you need at least three values, at least afair. One value doesn’t make sense and two values would result in a straight line, you can try “linear” mode for that and see if the algorithm accepts this.

The error output is quite raw and unpolished, there’s certainly room for improvement, sorry for that :slight_smile:

I’ve had the idea to read darktable’s database, to eliminate the need for globbing wizardry. But my To-Do list is already pretty long :sweat_smile:

1 Like

Hi there,

I also have a nooby question:

My files (raw and xmp) are located at:
~/Bearbeiten/Zeitraffer/2021_04_02/RAW

When I execute dtlapse:
dtlapse exposure --xmps *.xmp

following error appears:

Traceback (most recent call last):
File “/home/coco/.local/bin/dtlapse”, line 8, in
sys.exit(main())
File “/home/coco/.local/lib/python3.8/site-packages/dtlapse/dtlapse.py”, line 209, in main
params = get_params(history, iop.operation)
File “/home/coco/.local/lib/python3.8/site-packages/dtlapse/dtlapse.py”, line 76, in get_params
raise ValueError
ValueError

Files look like “DSCxxx.ARW” and “DSCxxx.ARW.xmp”. Keyframe “Keyframe” is added to around 45 images, included first and last image.

Darktable version is 3.0.1

Kind regards,
Coco

Another question:

is it possible that 692 files are too much?

Certainly not, I just do not get around to maintain the script anymore. It’s based on dt 3 (?) and probably not working well or at all with newer versions. Sorry!