A full Android FOSS raw imageing pipeline (tutorial)

Thank you for sharing this! Do you mind uploading same image SOOC or RawTherapee Default profile output to compare the results ?

I wonder if termux:widget could help tie this together into a GUI-ish app: GitHub - termux/termux-widget: Termux add-on app which adds shortcuts to commands on the home screen.

Sure! Here’s the image SOOC with no profile applied:

I looked into the widget, but not in too much depth. Seemed like you’d still have to enter the file names, so I wasn’t too sure that it could be of much use. But maybe I’m mistaken. I should download it and check out what it can do…

Just want to add here that the Termux widget WAS a great idea. I’ve made scripts dedicated to each of the main hald-CLUTs I like. I can just select them from the list of scripts in the widget, the terminal pops up and prompts me to select the file name, and then it processes the image, makes the output file (with a standard output name), and closes. Fast 'n easy!

1 Like

Here’s a screenshot of what the widget looks like on my home screen:

Here’s an example of one of the scripts (for Agfa Vista).

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/OpenCamera
ls *.dng
echo "Choose the input raw image file"
read rawf
dcraw -c $rawf | convert - haldCLUT/AgfaVista.png -hald-clut ${rawf%.*}_Vista.jpg
exit 0

You have to copy that script into $HOME/.shortcuts, and then it shows up in the widget (as in the screenshot above).

Here’s a screenshot of the script running:

Saves the output image with a descriptive name. Easy peasy!

Here’s the link for the Termux widget, if you are curious: https://termux.com/add-on-widget.html

1 Like

Could you possible use this: https://github.com/termux/termux-packages/blob/master/packages/termux-api/termux-storage-get

To use the android GUI to select the file?

1 Like

That’s bloody brilliant! Took a while to figure how termux-storage-get works, but once I did, it works like a charm. The secret is that you have to pre-make an empty temp file first, because all termux-storage-get does is opens a file picker, and then copies the file you choose to the output file specified on the command line, but the output file has to exist already. I just made a blank “temp.dng” with the new file function in Amaze file browser. Then, I modified the script like this:

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/OpenCamera
ls *.dng
rawf="temp.dng"
termux-storage-get $rawf
dcraw -c $rawf | convert - haldCLUT/AgfaVista.png -hald-clut ${rawf%.*}_Vista.jpg
exit 0

Now, you choose the script from the widget, a file picker dialog pops up, you select your input file, and then it processes it and saves it to an output jpg. Now it’s REALLY easy peasy! Almost like a “real” app, lol!

Oh, and you have to install the Termux API app from the play store or f droid, AND install the termux-api package with apt from within Termux. A little unintuitive, but once you do both, it works.

Sorry I keep thinking of these things one by one! :joy:

1 Like

Lol! Actually, termux-storage-get seems only recently added to the Termux API. It’s in the package, but not in the documentation.

I had the idea to have a file picker for both the raw file and the hald-clut, and then a text dialog to enter a name for the output, but I can’t get it to work. It seems that each action launched by the termux-api runs as a separate background process, but not like a typical shell background process. The wait command doesn’t work with them, so they all run at once, not sequentially. :frowning:

OK, latest stage of the script. Due to the way termux-api works, Termux has no idea about any Android processes that are launched, so it doesn’t know to wait until they are done before going to the next line in the script. Traditional wait command won’t work because of this - it’s not a shell background process. The best solution I found was to poll for change to the MD5SUM of the two temporary files we set up in advance for the raw file and the hald-CLUT.

So, set these two files up using Amaze as I mentioned above. Then, install the core utilities in Termux so we have the stat command. Do it this way: apt install coreutils.

Then, make the “filmsim.sh” script and copy it to the .shortcuts directory. Here’s the script:

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/Camera
rawf="temp.dng"
hc="haldCLUT.png"
rawfMDSUM=`stat -c %Y $rawf`
termux-storage-get $rawf
while [ `stat -c %Y $rawf` -eq $rawfMDSUM ]; do
sleep 2
done
hcMDSUM=`stat -c %Y $hc`
termux-storage-get $hc
while [ `stat -c %Y $hc` -eq $hcMDSUM ]; do
sleep 2
done
out=`termux-dialog -t "Output file" -i "Enter a name for output image file with file extension."`
dcraw -c $rawf | convert - $hc -hald-clut $out
exit 0

Now, when you run filmsim.sh from the Termux widget, first you pick the raw file in the first file picker, then you pick the hald-CLUT in the second file picker, and then you enter the name of the output jpg in the pop-up text dialog. After a while the code runs and the image is made! Pretty much a whole GUI pipeline to process all your raws with any hald-CLUT you like… Fun!

1 Like

Here’s a final (probably) version of the script with some niceties added and a little added functionality. I found out that you with termux-storage-get, you can access the SD card or external OTG storage. For me, this means I could process .orf raw files from my em10ii directly from an OTG card reader plugged into the phone. I realized that I could unlock this potential with a couple of code tweaks, and it could become a pretty powerful mobile photo studio to go from camera raw to processed JPEG, and then upload to social media with your phone…
So I updated to be able to add any raw file extension. I also added code to make and clean up the necessary temp files for the copied raw and hald-CLUT files. And some useful text is written to the terminal as you proceed through the steps. I will write up a brief tutorial and some install instructions, and then make a pull request to add it to the scripts git repository. I’ll include a sample raw and hald-CLUT or two…

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/Camera
rawext=`termux-dialog -t "Raw file type" -i "dng"`
rawf="temp."$rawext
hc="haldCLUT.png"
echo > $rawf
echo > $hc
rawfMDSUM=`stat -c %Y $rawf`
echo "Choose the raw file you want to process."
termux-storage-get $rawf
while [ `stat -c %Y $rawf` -eq $rawfMDSUM ]; do
sleep 2
done
echo "Choose the hald-CLUT you want to use."
hcMDSUM=`stat -c %Y $hc`
termux-storage-get $hc
while [ `stat -c %Y $hc` -eq $hcMDSUM ]; do
sleep 2
done
out=`termux-dialog -t "Output image file" -i "example.jpg"`
echo "Processing. Please be patient...."
dcraw -c $rawf | convert - $hc -hald-clut $out
rm $rawf $hc
exit 0

Awesome. If you want to make a hald-clut folder in your PR, I was thinking of adding one anyway.

1 Like

Cool. Will do!

Just an update on this. My phone (a nexus 5x) went into that catastrophic bootloop, and I had to get it replaced (this is the second time I’ve had to do this). It was under warranty, but I lost some of the images and hald-CLUTS I had on the phone but that were not backed up. It was only a few things, but these were the ones I was preparing for the tutorial. I’ve just gotten my replacement phone, but haven’t had time to get back set up. So, there will still be a little delay before the final tutorial and upload to the scripts repo.

2 Likes

D’oh! Sorry to hear it! Hope you can get up and running soon, and we’re ready to help if you need it!

Did you want to post the tutorial as an article on the main page?

Thanks Pat! I’d be happy to have the tutorial on the main site! Let me work up, and then you can take a look at it and determine if it’s good enough to be added…

A quick update. I have gotten a lot of the tutorial written. I’ve updated my fork of the pixls Scripts repo, but will wait to issue a pull request until the tutorial is complete. However, you can see what I’ve gotten done so far here: https://github.com/PixlsStuff/Scripts/tree/master/android_filmsim. Let me know how it’s looking so far!!

2 Likes

@Isaac This is awesome. Sadly, my phone is too old and unsophisticated to do this :sob:.

@afre You might still be able to do the Termux part! And, if all else fails, there’s always a time to get a new phone! :smile: :wink: