Lots of raws to process -- How to minimize impact on computer?

timeout is a quick and dirty way, giving you an option of delaying further execution by a maximum of 2.77 hours. I do not know if you can tie it to temperature analytics… :thinking:

Probably the best way is to schedule the processing so that it happens whenever you are sleeping and not using the laptop.

Last thing is to remember to redirect the standard output and error into a log, so you can keep track of what is being done and the errors that may surface.

Edit: Does ART have a built-in batch processing feature/tool? Calling the app repeatedly seems inefficient…

If you have decent jpg previews in your raw files you could extract them…

2 Likes

Yeah, maybe so. I know I can change process priority in Task Manager, but that would have to be set for every image (each is a new PID). But apparently one can (or at least could, in the past) run a command with:

start /low command line ...

I’ll run some tests on that when I have time. Hopefully soon, maybe…

[ Unrelated details ] I’ve been out of the loop for a bit. We just returned today from a Fri-Mon 1500 mile drive. My oldest sister in Kansas City passed away in January, but they were having subzero (Fahrenheit) highs for days / weeks, so any kind of gathering (requiring travel) had to be postponed. It happened Saturday evening and was really good. But we drove 750 miles Friday, 400 Sunday and then 350 today – with 80% of it in the rain. And 300 miles from home we had a tire start shaking, which fortunately we were able to get replaced promptly this morning (it was starting to split). Anyway, all’s well that ends well, but I’m in catch-up mode for a while now… And Wednesday we’re getting all our home window panes replaced, so there goes that day! :slight_smile:

Thanks.

1 Like

Well, there it is. Thanks!

Yes, it can be done in the GUI, but I was hoping to avoid the unneeded overhead of the UI (GTK, etc.) by using the command line. One advantage I suppose of using the GUI is that I could lower the priority using Task Manager. Offhand I don’t know if it spawns subprocesses, though…

Hmmm…

They’re full-res, but I want to raise the chroma a tad, add lens correction, etc. so I need to process. However if I could extract the thumbnail from the raws after applying my edits across the board that might work. I wonder if that would take less CPU than processing to a JPG?

And how is that done, by the way? :smiley:

I was thinking of this:

and this (from the “change plan settings” in the above screenshot
image
but I haven’t really looked into it. Will depend on hardware too.

Now that’s a road trip!

You can have a range of sizes in DT …not sure of the quality but there are some settings around that as well… in ART I would have to go digging…

There are other settings there that one could explore. I tend to resort to that only when resources are already low and I want to smooth the throttling and fans a bit more.


@lphilpot Sorry for the loss. Glad you returned home safely.

1 Like

Yeah, that’s really at the OS level although by default it would cap anything art-cli was doing. Ideally I’d like to constrain only art-cli, but I’ll have to look into it.

After working on Windows in IT for so long, it became second nature on every new (and often, not so new) PC to go into power settings and disable every sleep, limit and otherwise “green” feature we could find. Not that we wanted to waste power, but those settings always seemed to cause issues.

1 Like

Thank you.

Mm. Yes. I was kind of working on the assumption that you’d leave it running overnight, so you wouldn’t be doing anything else…

unless your laptop has cooling issues, you should be fine to run it.

if you’re really that worried about the laptop, you can probably spin up a VPS on a cloud provider for a few bucks, rysnc all the files over, process them, and rsync it all back without too much issue.

2 Likes

Hi,
From art’s pov:

  • don’t call ART-cli.exe separately for every file, you can call it once to process all files (if they are really many you can hit the limit of the command line length, I don’t know if this is an issue on windows though)

  • do you need the full sized pics or are you resizing them? If the latter, you can consider the fast export pipeline (-f flag) which can speed up significantly the process depending on the final output size

HTH

dcraw -e *.raw

1 Like

I initially was going to call art-cli.exe once on the directory, not the files individually. The only reason I ‘went’ multiple invocations was to potentially insert a sleep (or equivalent) every now and then. Per Google Windows 11’s command line length is 8191 chars total. So, if I have (~1061 images * 12 chars per filename) + params, it’ll apparently overflow. I could divide them up into a handful of directories easily enough, though.

Oops, I didn’t think about resizing. For my purposes HD (1920x1080) is good. So fast export will speed up resizing, but will non-resized images be still faster? Or is there a benefit to fast export regardless? We’re talking about 24mp (6000x4000) images.

Or, as @afre indirectly touched upon is there any advantage to using art-cli.exe vs just doing it in the GUI?

Thanks.

I have no reason to believe it does, it’s just a regular run-of-the-mill laptop. But if I found out the hard way I was wrong, that ounce of prevention might have been worth a laptop of cure. :slight_smile:

Thanks.

I wouldn’t be to worried about the computer, the CPU will slow itself down when it gets too hot. Depending on the orientation of the fans in the laptop you may want to elevate it to get more air flow under the chassis.

I was referring to the command line (@agriggio confirmed it is possible in his comment). Perhaps, batch files do not cause command line overflow. @snibgo may know more about command line and batch processing.

Batch operations from the command line are possible, just point art-cli.exe at a directory rather than at an individual file. The selection criteria for the batch op is then just whatever’s in the directory.

I do loads of batch processing, eg from frames of videos, mostly using ffmpeg and ImageMagick. I’ve never used art-cli.exe.

On my Windows 11 laptop, and previously with desktops, this leads to lots of fan-noise. This has never been a problem. Just ensure the vent holes aren’t blocked, eg by accumulated dust. If a job takes my computer all night to process, then it does, no problem.

The Windows start command can run a program or batch file at a low priority. If the computer isn’t doing other work at the same time, then this makes almost no difference to anything. Putting work at the back of the queue, when there is no queue, makes no difference.

Some software (eg ImageMagick) contains facilities to throttle CPU usage, and limit the number of threads. These can be set in policy.xml. These settings will reduce the CPU heat.

I can’t find a Windows setting that can, for example, limit a process to 25% of CPU usage.

If desired, pauses can be inserted between intensive commands in a batch job to allow the CPU to cool down. But if the fan still needs to run during the intensive commands, there isn’t much point. The temperature cycling may even be slightly detrimental to the hardware.

1 Like