Dear CarVac,
Thank you ever so much for your prompt and kind response. I truly appreciate this. I have invested so many days and nights, over the last year, getting to learn darktable and photography, and especially the dream of being able to take advantage of post processing of raw files, to improve upon some of what is possible via in-camera jpegs.
I will admit, it has been a bit of a frustration with darktable, the tool idea is great - flexibility, and the user interface and general stability of recent versions on Windows has been outstanding. In theory it works, but one has to judge by the final output, the images.
Not wishing to repeat what you already know, but just to have some context.
There are two or three key modules which I have used to transform raw files in dt to a visibe image, which one can revise further with other modules. These three are the base curve module, the various recent versions of the filmic-rgb module - aka filmic, and strangely there is also a crude method for generating this image using LUTS, in the 3DLUTS module.
LUT’s. :
The challenge with all these methods has been, a relatively inconsistent success rate, of any of the methods. The LUTS based methods, obviously requires images to conform to some of the methods for capturing video, such as sticking to the recommendations for IRE, skin tone - etc =, which are pretty strange and not readily applicable to photo cameras, which do not have any indicators or meters for the kinds of things monitored in video cameras. But with the right image strangely sometimes all you need is a LUT, and youhave something decent. some of my best image development in dt has been with using LUTs. strange but true but the method is not assured, too many rules that the image must conform to, and some uncertainty in my mind about the implementation of the 3DLUT module, related to colourspaces - which is not adequately covered in the user manual - very little information about it there.
Base Curves :
Using the base curve generally works, once you have an image that is definitely not overexposed, and ideally a bit underexposed, or dare I say it - properly exposed, according to the input ideal for the base curve. When I started using dt, I invested about a month with at least an hour a day, hand creating a set of base curves - conforming to different image styles, and exposures, for the digital camera which was my main camera at the time. Sure I have truly thrown away lots of time on darktable, cos in theory the whole solution is a great idea if it works. So I had a set of different custom base table curves and would try out different ones, as a starting point for each photo, and pick the most suitable and proceed from there.
The challenge with base curve approach is that from my laymans understanding, modifying luminosity leads to inconsistent changes in chromaticity or maybe its vice versa, anyway any attempt to keep things consistent then leads to a loss of colour and the image starts to look dull. The typical work around then is to allow the colour to become excessively saturated by turning off the luminosity preservation in the module - set this to none, and then attempt to use another module further down the process pipe, to reduce the oversaturated image. Clearly the end result of this approach is not consistent, across images. Definitely not point and click. It’s point click and pray, and up to 30 minutes of going back and forth between different modules trying to tame the beast that ran away and put it back in the cage, the beast of over exposure, created by turning off the luminosity preservation in base curve.
FILMIC
This has similar issues, by default, you get somewhat desaturated unnatural colours, but if you turn off the luminosity preservation option in this module, its similar to the base curve, colours become over saturated and you have to suppress this behaviour down the line bu undersaturating. Not a consistent approach cos the image also becomes unnatural. The filmic also has some issues with strangely modifying the sharpness of certain portions of the image. The way I express it best is to think that it destroys the natural contrast gradient in locally contrasted areas, and information or detail is lost. And it is impossible to then recover t
It was these inconsistencies with the most important aspect of photo editing, the initial “development” of the photo that I found really problematic, and this should be the easy part. An image editor should present me with a decent image, that is “high quality” from the start, without me having to first fuss over it. Then I can go further to edit to taste. Almost like buying a car, then I have to push it to start it - every single time. Trust me I have once had a car which behaved like that for a few days, when the electrical/battey charging system went bonkers !!!
All this was what led me to filmulator, I opened a few images and was impressed from day one. Definitely something I could work with, and develop further.
I appreciate that filmulator encapsulates what would be equivalent to several of darktables pipeline modules, into one tool, and implementing this in darktable would require fragmentation of the code into probably several modules…
I have read you response and here are my thoughts.
Especially in the open source world, its best not to reinvent the wheel.
For example you have a librtprocess library that is reused in rt, and in some other apps, , and this is also reused in filmulator, which is an excellent idea - develop once. From a brief look this module addresses the demosaicing of raw files.
Your app also does other things, provides a user interface, and organises and generates tiff and jpg files. On that note one of the major issues I found out when I gave your filmulator app a real workout, was the inability to determine the properties of the generated image file. I could not set things like jpg compression, bit depth, of the exported image file, nor could one address things like the gamut/colourspace of the final product. These are not particularly weaknesses of filmulator, cos the real strength of the app was to demonstrate its “film like image development”, which it does in a very promising manner.
I apologise if I am very direct, considering the time in which filmulator has been in development, it may take a while for the app to add all the bells and whistles, such as being able to change the settings for exported jpg files.
The beauty of something like darktable is that it already has stable code for these “housekeeping” functions like creating and generating and setting options for the kind of file to be generated., But it is not the best for presenting a simple high quality image to the end user as a starting point for further editing of the photo, which can be quite discouraging, and that is where your skills and what you have achieved could be the saviour of darktable.
I.e you being able to add much needed improvements to darktable, and your inputs can take advantage of existing house keeping code that already exists in darktable.
Lets take it one step after another.
Your filmulator app, appears to have what I would call several key aspects.
-
Debayering/Demosaicing. Which is possibly different from that in darktable. imagine you took that aspect of your code, and developed an alternative debayering/demosaicing module, or your code/library became available as an option in the preexisting module in darktable, so that one could choose settings or simply use the defaults that you already use in filmulator (should it not be expedient to expose such settings to the end user who could end up doing more damage than good - one does not give knives to children - cos they may harm themselves!).
-
Your app has an excellent approach to establishing the “film curve”. and you could create a module for this feature that darktable, in spite of all its efforts so far, has not excelled at, that encapsulates the key aspects of this film curve as an alternative to the filmic or base curve. I took one look at the image generated by default in filmulator and said yes - this is what we need in darktable.
So the idea is to pick the battles carefully, to focus on one or two modules that you could add to darktable, so people like me can finally enjoy darktable.
It does not have to be encapsulated in a single module. I’m sure you understand what I am trying to say.
Filmulator is a great idea, but it will take a lot of work to add all the user oriented features that would complement it, such as the generation of different image types, which is already included in darktable.
I nevertheless think adding the key aspects of filmulator to darktable, would be the most rewarding investment of your available energies.
- You would get a huge user base of existing users - a huge one, to challenge and test all your ideas. In recent times I have had to go back to using tools like Capture One Express for Sony, cos like filmulator the image development by default is just better.
I appeciate that one of the things you are considering, is the inclusion of a noise reduction module, and if you have alternative ideas about this, that could also be a new module in darktable, which users can optionally choose. This is the beauty of darktable, the potential flexibility for the users and also for the developers, so each of us concentrates on what we do best.
Those who know how to develop code for image processing can concentrate on those aspects, while others who enjoy improving user interface can also concentrate on that, and everybody benefits.
Please give my suggestion another thought, and I do hope I can encourage you to adopt the suggestion. Sure it will take some work, to conform your app components to dt’s , in separate modules, but especially as you do mention that you have done some of this before in dt, it should be a bit easier for you.
I look foward to a positive response and the hope that your existing efforts can become one of the next step forwards for darktable.
I acknowledge - nothing progressive is easy, but the effort is worth it.
Thanks for thinking through my suggestion.
I was especially appreciative of the excellent and not excessive sharpening in your filmulator tool. dt needs such quality of image sharpening. Lots of ideas from your app that could enhance dt, if implemented in appropriate new modules, as alternatives to some of the existing modules in dt.
Regrttably I have had to stop using filmulator, cos I cannot use the exported jpg, as I cannot define its quality, size, etc, or export into other image types like png, and all these things are already in dt. As you can see - merging what you have already achieved with what has already been achieved in dt, is the best way forward for both apps. - the features of filmulator and the excellent processing infrastructure of dt.
Hope that’s all we have. And we live only once.