Eagle-eyed viewers may note the new presentation of devices.
Under the hood the biggest change has been code to generate thumbnails asynchronously and in parallel, using as many cores as the CPU has, up to 4. Working in parallel certainly helps speed aspects of the overall thumbnailing operation up, which is important for users who download thousands of files at a time. Not every aspect of thumbnailing can be made parallel, however. Reading the files from the drive/camera must be linear, because this part either cannot be made parallel (libgphoto2 limitation) or reading files in parallel results in lower performance because of disk thrashing. This bottleneck is not easy to work around.
The UI now explicitly differentiates between downloading from devices / memory cards / cameras, on the one hand, and a folder on the computer, on the other. I think that will make it easer for new users to figure out what’s going on.
The checkboxes beside a device and a folder This Computer can be used to select / deselect photos/videos.
The toggle switches beside Devices and This Computer turn on and off scanning/downloading from that source. I hope that will be obvious to users. Curious fact: Qt widgets provides no toggle switch, although QML does. So I had to wrangle a slider into what you see there
If you want to download from both devices and a folder on your computer at the same time, you can. My guess is that the number of times anyone would want to do that is very small. If it confuses people, I may have to artificially limit that. Let’s see.
You can select cells in the Timeline and it will filter the display of thumbnails to show those only from that time period / day / month. You can of course select multiple cells. too. The one thing I still need to figure out is how to best handle deselect – that is when the user wants to see all of the photos, not just those from a particular time range. I’m not sure if I need to add a special button and make it prominent in the UI, or some other way, perhaps if the user clicks in a cell that’s already selected, it becomes deselected.
You can turn off individual panels, to maximise the thumbnail display area. Here both left-side panels are turned off:
I’ve only just started to use Rapid Photo Downloader and as I have the latest version there isn’t a GUI way to set the filename template. This sort of defeats the object for me and is a major block - I have found the config file but there doesn’t appear to be any info on what the line for how a filename can look on your website.
While not a ‘feature’ could you let me know what the textual options are for renaming photo files (i.e. can it use info from the exif data etc).
I understand that this issue will be sorted out once you have enabled the filename rename setting part of the GUI but I ( and any new user) really can’t start to use your program without this info.
Hi, you can import program preferences from version 0.4.x into 0.9 alpha. Set your renaming rules in 0.4.11, for example, and then import them into 0.9.
Or else you can wait until I release alpha 5. The file renaming GUI has been written:
I’m doing the final code on displaying the example file names, which involves using a sample file from the download source. ETA is unknown right now as I just (1) changed country and (2) it’s the start of a new semester, hence I’m extremely busy.
If you like you can always download the latest code from launchpad. The sample file names issue is only for display purposes.
Thanks for the fast response. Hope your new location is as good (or better than) the old one! To be honest I’m not in a great rush - just wanted to see your software working in all it’s glory. I’m currently just copying the SD card across to a ingest folder in the laptop and running a series of exiftool commands to rename and place the files in my target photo external drive.
Enjoy the new semester and I’ll await the next version!
I hadn’t seen this thread before, hope it’s not too late to add my two cents I haven’t tried the alpha yet (is there a PPA?), so forgive me if some of this is already implemented, but I figured I should brain-dump before my expectations are influenced by the existing work.
I think these two points are good:
One of the very annoying things about 0.4.1 is how hard it is to select “only the last shoot”. We can’t rely on rpd to keep track of which files have already been downloaded, since we often end up renaming or maybe moving the folder after import (e.g. it downloads to ~/Photos/2016/2016-09-18 and then we move it to ~/Photos/2016/2016-09-18_birthday or ~/Photos/work/in-progress/2016-09-18_jamie-and-lees-wedding). But of course, this means having to do a lot of fiddly clicking on those tiny boxes, and there’s some bug or something which means the shift-click doesn’t work (or I’m not smart enough to figure out the UI), which means manually selecting, which means we end up using the file manager instead of rpd for those times when we’ve not emptied the memory card in between shoots.
So groups would help here, as would having a UI for selecting photos that corresponds more to how regular file managers do it. Before I found out you had to click the little box of one photo after clicking the photos themselves, I tried clicking the photos themselves and then hitting the “select all” button below, which of course did not do what I wanted. My girlfriend did the exact same thing. So the whole “two kinds of select” thing is confusing.
If you’re doing groups anyway, perhaps a speedy method of showing what’s on the card before downloading would be showing just the first 5 photos of each group, and then elide the rest, like
2016-09-12: (201 more in this group, click to load thumbs) 2016-09-19: (697 more in this group, click to load thumbs)
I don’t care about having rpd do backups (I have my own backup scripts), though having the ability to run a self-chosen command after import sounds like a good idea.
Regarding having separate download locations – I think it’d be hard to do this in a way that will be fast enough. We typically just want to get things in as quickly as possible to stop worrying about losing the card or whatever. But I suppose an easily accessible toggle somewhere between some presets might be useful (e.g. I can set up a work-preset and home-preset, with differing download folders and perhaps different after-import-scripts).
One thing that’d be nice is having a per-shoot format string in the download folder – is this what you mean by Job Code? E.g. setting the destination to ~/Photos/%Y/%Y-%M-%D_%P where %P is from a prompt showing after I hit the “Download” button (and where it’s OK to leave it empty). Or just a text field always showing right above the download button.
“Remember files that have already been downloaded.” – this won’t work if I move the folder, right? Or does it store exif info in some database?
This thread was started almost 12 months ago. There has been much development since then including several alpha releases, accompanied by often extensive release notes. Let me be abundantly clear: it’s much more productive, helpful and sensible to comment on the work I’ve actually implemented than to do a “brain-dump” that ignores all that.
The alternative is checking / marking a photo as soon as it’s selected, which is problematic for several important reasons which I won’t go into here.
I put a lot of time into the documentation for the version you have (0.4.x), and have started documenting the alpha release. The documentation contains the answer to this question.
I’ve been playing with RPD 0.9.0a4 this week, and I am really impressed with the general layout and flow. So far, I’ve tried it with a Samsung Galaxy S5, a Canon point and shoot, a Nikon D7000, and a SD card removed from the Canon, and it has worked well for all of them (other than the known hassles you documented for the phone). The timeline with adjustable proximity setting is a super way to find the logical breaks between sessions. Once the job code is implemented, I think RPD will do exactly what I believe I need. It tries to do one thing well, rather than trying to do a mediocre job of a whole bunch of things.
I’d be interested to know how you plan to implement the job codes. The implementation probably matters more when importing a lot of historical work than it does once you’re settled in and just downloading a couple of sessions at a time. Personally, I think being able to do something like right-clicking on a grouping in the timeline to access a control to set the job code for that grouping would be a good way to go. That would allow you to label a large number of jobs, start the download, and just let it rip.
Two non-defect things I found less than ideal:
I found the thumbnail size to be barely large enough for me to identify what a session was. It would be nice to have a setting to customize the thumbnail size (not necessarily freely, just having a few choices would do).
I did not find a reliable way to select all the photos from a session, and only those photos. Clicking a session in the timeline caused only the photos from that session to appear as thumbnails, but doing common things like selecting the first of these and shift-selecting the last of these did not have the desired effect of selecting the entire group. I would either wind up with only those two photos selected, or the entire contents of the camera or card. This may be ignorance on my part, but I could not locate doc for selection and tried a lot of things and wound up individually selecting each thumbnail.
Overall, great package, and I’m looking forward to trying out the next alpha release.
EXIF thumbnails are 160x120. That’s a real limitation in the sense that generating thumbnails larger than that makes generating thumbnails a lot slower, particularly but not limited to when downloading directly from cameras. Handling larger thumbnail sizes also makes the thumbnail generation code quite a bit more complex, which makes the code harder to write and harder to maintain, and thus delays the release. So it’s firmly in the “nice to have one day in the undetermined future” basket of goodies.
Works for me. Keep in mind the “Select All” checkboxes in the bottom right of the window.
If you can reproduce an exact set of steps to reproduce a bug, then please file a bug report. It’s possible there is a bug because the code driving the selection logic is surprisingly complex.
I’ve filed a bug for the selection issue I ran into, and another one related to selections affected by the granularity of the timeline. While I was recreating the selection problem, I encountered a couple of other selection issues as well. Would you prefer me to add those other scenarios to the bug I already created, or create a separate bug for each one?
Have been using the tool for the last two days in order to improve my workflow which up until now included mainly:
cp/find + exiftool + darktable
This is my feedback for v0.9.0b5:
Rapid Downloader as Categorizing and Downloading:
Bug: Slashes in job names create directories
Bug: Above 1000 downloads does nothing / breaks up
Bug: Subseconds is broken, if tag is missing returns nothing (and not 00 as documented, though I see nothing as a better solution)
Req: Queue multiple downloads while freeing the GUI for more work.
Req: Remember applied job names after application restart (currently is lost and task needs to be redone)
Rapid Downloader as Renamer:
Req: Parallelizing Download and Renaming. Here I am not sure but I have the feeling next file does not start to download
until previous file is renamed.
Req: The same way one can create and manage Job Codes, one could manage tag perset formatings (ie: my own date format availible as one entry from the pull down menu)
Req: Extend the automatic _1 _2 automatic suffixing into its own tag, ie <samnanecounter (two digits)>. That we the user decides where it goes and how it is formatted
Req: Non existing tags should return nothing.
Req: Add Camera model to Videos
Req: Offer simpler configuration, where Videos and Pictures have the same destination and renaming strategy.
Req: tag grouping to use the first existing tag: ie () or ()
Req: Conditional text, ie: add dot only if subseconds defined: (.)
Req: Add many more tags. To avoid clutter there could be a pick up window to add user specific tags
After two days of testing I have decided to use RPD to download files this way: < DESTINATION >/< YYYY.MM.DD > < Job Name >/< OriginalName >
This helps me to find, chose and download all what it needs to be downloaded and has not yet been downloaded into a themed and dated directory.
I continue with exiftool (hopefully RPD would implement stuff to achieve similar results(
Exiftool renames the downloaded files to:
Albums/< directory name as created by RPD >/< DATE as YYYY.MM.DD HH.mm.ss >.< ## > < MODEL >.< ext >
where
DATE is taken from the first match of:
$DateTimeOriginal
$CreateDate
Uknown (This will tell me to inspect such file with exiftool to find another Tag containing date information)
<##> is taken from the first match of:
$SubsecTimeOriginal
$ShotNumberSincePowerUp two digits zero leading
Exiftool generated index counter of identical file names, two digits leazing zero, fallback if no info found in metadata
MODEL is taken from the first match of
$Model
$DeviceModelName
Uknown (This will tell me to inspect such file with exiftool to find another Tag containing model information)
Cheers, Ariel.
P.S. My Timeline looks different (bug?). I attach screenshot
Thanks for your feedback Ariel. Judging by the screenshot I guess you’re using it to rename files that you’ve already downloaded and possibly edited. Is that correct? I mention that because it appears possible the date time metadata in some (many?) of your files is mangled. I suggest that because the Timeline should be dislaying correct time values, but it’s not.
Please note if you’re intending to use the program as a general purpose file renaming tool for files you’ve already worked on, you should probably be using another tool. See the FAQ, which I updated today: Rapid Photo Downloader: Frequently Asked Questions (FAQ)
Some observations:
You cannot use a forward slash in your filenames, just like you can’t have a filename of only a dot or two dots. Filename - Wikipedia
What do you by it’s “broken”? Are you saying your photos have subsecond metadata but the program is not extracting them? I’ve updated the documentation to remove the reference to ‘00’ being added , which is a holdover from the legacy version, and no longer valid.
Not a chance, sorry. On the to do list is save presets as a download profiles.
An interesting idea. But what is the use case for it? Why would you need it?
You can already create and save your own file renaming and subfolder preferences, which makes adding them to the combo boxes in the subfolder generation / file renaming editors redundant.
The program provides a variety of sequence numbers to generate unique filenames. Use them if you like, or not, it’s up to you.
Sorry I don’t understand what you mean here.
It’s a nice idea, but getting the metadata from which tool, ExifTool or libmediainfo? Which is more reliable?
Not a chance, because the metadata between photos and videos is different, and because historically, users have specifically requested they be different, including for backup destinations.
That seems complicated to implement in a GUI, and I’m not about to rewrite the existing name editors given the huge amount of work it was to write them as they currently exist. But more generally, I’m not aware of any GUI program that provides such a feature. Do you (or anyone else) know of one?
It’s an interesting idea for a future version. But actually implementing it in a GUI so that the majority of users are not confused strikes me as rather tricky, so I’m not sure at this point if it would ever be implemented.
More generally, given your use of conditional tags, which are far easier to configure using a non-GUI tool, my impression is that Rapid Photo Downloader is not the correct tool for your needs.
By “tags” I assume you mean metadata. Which metadata, precisely?
Not really, I tried to document something I saw while downloading which looked similar while checking ms SD card again (that’s the screenshot; not trying to use the tool for further renaming, I used exiftool for that). And both look different from the above posts.
You forgot to include the previous line
“File system utilities and naming conventions on Windows prohibit…”
In unix we are more forgiving.
Your tool accepts “:” in the job name (also prohibited in that list) and expands properly.
It also accepts the “/”, but it does not expand and creates directories instead.
If accepting the / in the job name is a feature for creating sub-directories, document it as such.
If it was an unexpected behavior, reject it or let it expand as part of the directory or file name.
Currently is a confusing behavior.
I see this was a documentation bug that you just fixed, thanks
These are two altogether different things.
Currently I can select a bunch of pictures, tag them with different job names and press download (in my wifi case takes a long time)
While it is downloading I am able to continue and select another bunch of files. Here I need to wait. When the download is over I can apply a job name to the already selected pictures. It would have been great if I would have been able to apply job names and continue selecting and applying job names while downloading. And if I could queue these new selections for downloading and then go for lunch even better.
Similar to the above response. If I apply a Job name and for for whatever reason the download is not clicked, all that cataloging work is lost. It happen to me a few times. We can either fix the user or make and improved GUI
All the time presets you provide are redundant too for that all matters, users can build them on their own with the basic blocks. It’s all about convenience, but the same can be achieved without it.
The program provides some sequence numbers for series of pictures. Download series.
It also provides a rudimentary index number for duplicate names (ie: 3 pictures within the same second) I proposed to make this index number (lets call it dupindex) more versatile and consistent with the rest:
the same way you have < Video number (All digits) > or < Video number (two digits) >, you could place this variable into < dupindex (two digits) > and let the user place it and format it as pleased. In exiftool there is something identical and very useful with lots of adepts.
I like Exiftool, but I cannot tell you from where your implementation needs to take it. But I will expand on this below.
Yes and no. In exiftool I can use (full path) $exif:originaltime or $originaltime. The second will pick up ANY originaltime, even if it comes from alternative XMP or quicktime tags. If found, it is expanded, if empty, it returns an empty string.
In the iPhone era, I guess many people would also like to have all their pictures and short videos together and sorted chronologically, one directory, one naming convention, Of course we can say this is for more sophisticated users which will not mind to have identical configurations for pics and videos if they mind so.
FileBot is using it, it inherits from groovy.
I imagine you could add OR/AND/NOT and some sort of grouping to the GUI.
FIleBot is similar, with a GUI that in fact creates a text definition, which could also be entered and edited manually.
For the GUI, you could define some Conditional Text, which will expand only if next tag returns true.
Let me see my FileBot definition for movies:
{mTitle}{’ 3D’+s3d.upper()} ({y})
in this case, mTitle will expand only it it exists, ’ 3D H-SBS’ will expand only if s3d tag exists and (1998) assumes that a year will always be returned.
Of course I could list the metadata I particularly use like (let me check my script) for ANY type of file, Video or Photos:
DateTimeOriginal
CreateDate
FileModifyDate
SubSecTimeOriginal
ShotNumberSincePowerUp
Model
DeviceModelName
but I do not know, I add dynamically new tags as needed (I recently found that a camera had no SubSecTimeOriginal but it did have ShotNumberSincePowerUp, which provided an analogous outcome. Your tool could accept user added metadata definitions. Like giving me the chance to add DeviceModelName myself. This would be probably the most powerful addition from all of what I have mentioned so far.
I’m not going to reply to each and every item because it takes too long. Some comments:
I’m the one who answers emails from users who are trying to figure things out. I’m the one who writes the code. This project is over 10 years old, so I’ve been doing it for a while. The program is designed the way it is because I’m trying to meet the needs of a very wide range of users, from the most simple to the most sophisticated. When I say there is “no chance” of changing the fundamental design of the program so that it can be used a different way, I mean it. I will not write at length justifying and explaining those design decisions.
/ cannot be used as a filename character in Unix like systems. Reread the link.
I cannot diagnose bugs without a proper bug report, including the program’s log files and an adequate description of how to reproduce the problem, including the source files if necessary. If you’ve found a bug, file a bug report.
Cool down. It’s your software, it’s free. You do whatever you want with it, You explicitly asked for feedback. My way of paying you is providing my feedback. As you put it “reply to each and every item because it takes too long”. You can see the length of my answer, I invested a good deal of time to provide feedback. Take what you want, trash the rest. You do not need to justify yourself explaining how supportive you are.
I think the program would be more user friendly if you rejected the / in the job name, or if you alerted that sub-directories will be created. Job names are created all the time, and if someone would put one there by distraction. will discover after the fact that files are not where intended but in a structure that will need to be undone and fix it. I am one fool who bore the consequences.
As I said before, bugs I was going to do it properly through the bug tracker. Comments about bugs are here are just “in case it is trivial and you did not see it”. To report the download problem for 1000+ files and others bugs properly requires a good deal of time. But I believe I just lost my will to spend more time on this.
Ariel, you misunderstood my message. I literally don’t have time to write an extensive reply to the type of feedback you’re giving and the requests you are making. I don’t know how to make it any clearer.
Although I’d never heard of FileBot before, much less used it, my impression based on its screenshots is that its renaming system is more akin to writing a regular expression than the kind of renaming tools available in Rapid Photo Downloader (or Lightroom or Photo Mechanic or other tools). Which is fine. Some users want that extra power and flexibility. That’s why I suggested ExifTool is the better tool for you. Generally speaking, ExifTool is the best tool for some jobs. Exiv2 is better in some other circumstances. Rapid Photo Downloader has its niche too.
Thanks for a wonderful tool. I’ve just started to use it regularly.
There seems to be an operational anomaly on startup.
When I start the program without a camera attached to the usb bus it works. When I then attach the camera it see the camera.
but
if the camera is attached before startup it fails to startup. I have filed a bug report in the launchpad system with log file and error. Bug #1695012 “initial startup fails when camera is attached” : Bugs : Rapid Photo Downloader
I haven’t tried the alpha yet (is there a PPA?), so forgive me if some of this is already implemented, but I figured I should brain-dump before my expectations are influenced by the existing work.
2016-09-12:
(201 more in this group, click to load thumbs)
