When you archive, could you just add some of your own compression? lz4 or zstd or even just zip each file? like .arw.zstd ? A bit of a PITA indeed, but as someone who also has 100 MB plus file sizes now, it does eat up the storage way more quickly 
1 Like
Iāve not tried that yet, and so far I havenāt archived a single photo. I just keep filling up my storage and doing occasional culls to keep total storage manageable (currently about 1TB). Conceptually, I donāt really want to archive anything because I like all my photos available to view and edit at any moment. I tend not to ever āfinalizeā any photo, although it is something I want to start doing as Iām considering printing and displaying certain photos.
But Iām certainly open to new approaches if someone can convince me itās a better way. I was close to getting an a7R V just for the lossless compressed RAWs, but I couldnāt quite justify an extra $1000 for a different file format!
1 Like
But thatās like two not very large hard drives with prices how they are right now 
There are some file systems that you can setup to compress and deduplicate, like btrfs or zfs.
1 Like
Oof, I know. Maybe it would have made more sense to just get the newer camera. But I didnāt know the DNG issues until I tried it. I had thought it was a really good solution, and itās almost great. The files are essentially the same while being half the size. And if my lenses are in Lensfun, I can use that correction. Itās just the Tamron 50-300 that isnāt.
Iāll look into those file systems. Thanks for the tips.
And hopefully storage prices come down again in a year or two. Although that might be wishful thinking.
1 Like
Are you sure dng creator isnāt doing lens correction?
Also there is the FOSS dnglab, not sure if youāve tried that.
Pretty sure because I did some tests in the early days, although now youāve got me wondering again. Iāll have another check.
No I havenāt tried dnglab, but Iāll give it a go.
1 Like
Just looked again at some old test shots I did, and I can confirm that the Adobe DNG Converter does NOT apply distortion lens correction. It applies a very slight crop but no distortion is corrected.
After using the Lens Correction module in āmetadataā mode on the DNG file, TCA is applied but not distortion; whereas on the ARW file, both TCA and distortion are applied. So it seems the distortion metadata is contained in a Sony format that DNG strips away, or it converts it into a format that Darktable canāt decode.
If anyone wants to have a play:
DSC00385.ARW (117.7 MB)
DSC00385.dng (55.3 MB)
These files are licensed Creative Commons, By-Attribution, Share-Alike.
1 Like
Iām useless when it comes to image-processing algorithms, so I donāt think I can do much with the code. But good to see at least some minor progress.
I have some updated values for the 24-70 f/4 S, but sending them to lensfun (Github issue) was apparently declined? Idk. I was asked to provide photos with requirements that suit the calculation itselfā¦ but I already had the new values calculated. The issue was closed shortly after (I was planning to supply photos later on) so idk, how does contributing to lensfun actually work? 
Downloaded the Windows exe but canāt seem to get it to run, even in Administrator modeā¦
Where did you get the values from though?
Lensfun implements a few different lens correction algorithms, but none of them are the same as what Nikon uses for their lens correction, so if you just grabbed values from your raw file, that isnāt going to work, thatās why they asked you for the raw files themselves, theyāll run and validate the calculations for you. They did that pretty fast for the GR IV shots I submitted recently.
Erm, my method was a bit strange. Iāve followed a tutorial using Hugin Panorama stitcher to get some a, b, c values by marking straight lines
(I have the video of the tutorial downloaded, but idk if I have the link the original, I think it was on Vimeo). The video is 32 megabytes so I could share it here if itās okay.
EDIT: bingo
After that I manually fine-tuned them so that they matched SOOC JPEGs as closely as possible. The result is noticeably better than the existing lensfun coefficients ā especially 35mm that bulged out in the center ā thatās I how I found the values are not right.
So yeah, I wanted to submit these values to lensfun so that I donāt have to manually rewrite local lensfun XMLs every Darktable update.
ah I see. Usually when you use the scripts, they make a PDF or some images with some graphs they can use to help verify them. I am not super surpised that they didnāt take just some numbers.
If you have the files, you should submit them 
1 Like
I would submit the RAWs if they didnāt have the reflection of my face in a window Are uncorrected JPEGs enough? I could censor my face like that.
I was planning to retake them later on, but the thread has been closed quite fast, so yeahā¦
I was planning to, but I couldnāt make it work on my system I think. Thereās still a tab with lens_calibrate.py in my browser for like half a year, but I canāt use it.
Pretty sure they want raw files.
Welp, new photos it is then. Gonna take a while before I get to (and feel like it) photographing a suitable building nearbyā¦
1 Like
I have a copy board with lots of straight lines on it. If I can get close enough to fill the frame, could I use that for taking my own lens readings for Lensfun?
I heard thatās not recommended unless you plan to use the lens primarily for close-ups, because distortion differs by focus distance. You need to shoot something at a distance of several meters ā I guess that depends on the focal length. That makes buildings with straight lines just about the only viable choice for good profile creation.
2 Likes
Yeah, I had a feeling that was the case. It means a trip downtown for me, which is something I only do a few times a year!
Yep, gotta do the same once I get the chance
Iāve got a few buildings that may serve me well (no reflections of me in the windows)
In the past, Iāve found by the time I get around to this kind of thing, someone somewhere has already submitted the lens data so I donāt need to bother. Iām kind of hoping that happens with the Tamron 50-300, which I would have thought is a fairly popular lens.
My other hope is that a darktable dev manages to decode the DNG metadata for lens distortion because Iām fairly sure itās in there somewhere.