A full Android FOSS raw imageing pipeline (tutorial)

Could you possible use this: https://github.com/termux/termux-packages/blob/master/packages/termux-api/termux-storage-get

To use the android GUI to select the file?

1 Like

That’s bloody brilliant! Took a while to figure how termux-storage-get works, but once I did, it works like a charm. The secret is that you have to pre-make an empty temp file first, because all termux-storage-get does is opens a file picker, and then copies the file you choose to the output file specified on the command line, but the output file has to exist already. I just made a blank “temp.dng” with the new file function in Amaze file browser. Then, I modified the script like this:

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/OpenCamera
ls *.dng
rawf="temp.dng"
termux-storage-get $rawf
dcraw -c $rawf | convert - haldCLUT/AgfaVista.png -hald-clut ${rawf%.*}_Vista.jpg
exit 0

Now, you choose the script from the widget, a file picker dialog pops up, you select your input file, and then it processes it and saves it to an output jpg. Now it’s REALLY easy peasy! Almost like a “real” app, lol!

Oh, and you have to install the Termux API app from the play store or f droid, AND install the termux-api package with apt from within Termux. A little unintuitive, but once you do both, it works.

Sorry I keep thinking of these things one by one! :joy:

1 Like

Lol! Actually, termux-storage-get seems only recently added to the Termux API. It’s in the package, but not in the documentation.

I had the idea to have a file picker for both the raw file and the hald-clut, and then a text dialog to enter a name for the output, but I can’t get it to work. It seems that each action launched by the termux-api runs as a separate background process, but not like a typical shell background process. The wait command doesn’t work with them, so they all run at once, not sequentially. :frowning:

OK, latest stage of the script. Due to the way termux-api works, Termux has no idea about any Android processes that are launched, so it doesn’t know to wait until they are done before going to the next line in the script. Traditional wait command won’t work because of this - it’s not a shell background process. The best solution I found was to poll for change to the MD5SUM of the two temporary files we set up in advance for the raw file and the hald-CLUT.

So, set these two files up using Amaze as I mentioned above. Then, install the core utilities in Termux so we have the stat command. Do it this way: apt install coreutils.

Then, make the “filmsim.sh” script and copy it to the .shortcuts directory. Here’s the script:

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/Camera
rawf="temp.dng"
hc="haldCLUT.png"
rawfMDSUM=`stat -c %Y $rawf`
termux-storage-get $rawf
while [ `stat -c %Y $rawf` -eq $rawfMDSUM ]; do
sleep 2
done
hcMDSUM=`stat -c %Y $hc`
termux-storage-get $hc
while [ `stat -c %Y $hc` -eq $hcMDSUM ]; do
sleep 2
done
out=`termux-dialog -t "Output file" -i "Enter a name for output image file with file extension."`
dcraw -c $rawf | convert - $hc -hald-clut $out
exit 0

Now, when you run filmsim.sh from the Termux widget, first you pick the raw file in the first file picker, then you pick the hald-CLUT in the second file picker, and then you enter the name of the output jpg in the pop-up text dialog. After a while the code runs and the image is made! Pretty much a whole GUI pipeline to process all your raws with any hald-CLUT you like… Fun!

1 Like

Here’s a final (probably) version of the script with some niceties added and a little added functionality. I found out that you with termux-storage-get, you can access the SD card or external OTG storage. For me, this means I could process .orf raw files from my em10ii directly from an OTG card reader plugged into the phone. I realized that I could unlock this potential with a couple of code tweaks, and it could become a pretty powerful mobile photo studio to go from camera raw to processed JPEG, and then upload to social media with your phone…
So I updated to be able to add any raw file extension. I also added code to make and clean up the necessary temp files for the copied raw and hald-CLUT files. And some useful text is written to the terminal as you proceed through the steps. I will write up a brief tutorial and some install instructions, and then make a pull request to add it to the scripts git repository. I’ll include a sample raw and hald-CLUT or two…

#!/data/data/com.termux/files/usr/bin/sh
cd storage/dcim/Camera
rawext=`termux-dialog -t "Raw file type" -i "dng"`
rawf="temp."$rawext
hc="haldCLUT.png"
echo > $rawf
echo > $hc
rawfMDSUM=`stat -c %Y $rawf`
echo "Choose the raw file you want to process."
termux-storage-get $rawf
while [ `stat -c %Y $rawf` -eq $rawfMDSUM ]; do
sleep 2
done
echo "Choose the hald-CLUT you want to use."
hcMDSUM=`stat -c %Y $hc`
termux-storage-get $hc
while [ `stat -c %Y $hc` -eq $hcMDSUM ]; do
sleep 2
done
out=`termux-dialog -t "Output image file" -i "example.jpg"`
echo "Processing. Please be patient...."
dcraw -c $rawf | convert - $hc -hald-clut $out
rm $rawf $hc
exit 0

Awesome. If you want to make a hald-clut folder in your PR, I was thinking of adding one anyway.

1 Like

Cool. Will do!

Just an update on this. My phone (a nexus 5x) went into that catastrophic bootloop, and I had to get it replaced (this is the second time I’ve had to do this). It was under warranty, but I lost some of the images and hald-CLUTS I had on the phone but that were not backed up. It was only a few things, but these were the ones I was preparing for the tutorial. I’ve just gotten my replacement phone, but haven’t had time to get back set up. So, there will still be a little delay before the final tutorial and upload to the scripts repo.

2 Likes

D’oh! Sorry to hear it! Hope you can get up and running soon, and we’re ready to help if you need it!

Did you want to post the tutorial as an article on the main page?

Thanks Pat! I’d be happy to have the tutorial on the main site! Let me work up, and then you can take a look at it and determine if it’s good enough to be added…

A quick update. I have gotten a lot of the tutorial written. I’ve updated my fork of the pixls Scripts repo, but will wait to issue a pull request until the tutorial is complete. However, you can see what I’ve gotten done so far here: https://github.com/PixlsStuff/Scripts/tree/master/android_filmsim. Let me know how it’s looking so far!!

2 Likes

@Isaac This is awesome. Sadly, my phone is too old and unsophisticated to do this :sob:.

@afre You might still be able to do the Termux part! And, if all else fails, there’s always a time to get a new phone! :smile: :wink:

I took a look at Termux app in the Play store but I wasn’t sure if it is actually an open source app. Are there any open source alternatives to Termux ?

It is open source; you can install it from fdroid, that’s where I got it from.

1 Like

@Andrius Yes, it’s open source: https://termux.com/source-code.html

All source code is available on GitHub, which you can access from links on that page…

I’ve opened some issues for termux, that may be useful:

@David_Tschumperle perhaps you can help here:

@houz perhaps you can help here:

And this is issue interesting too:

Thanks @Tobias! Perhaps we can all work together to get some if these things into Termux!

Help with what? Creating a dt package? I have no idea what this termux might be.