Hi!
Why isn’t there a pip installer? It would be great to be able to just do pip install rapid-photo-downloader anywhere Python exists! 
Take a look through the install script and it should be obvious why…
do you mean this ?
MO_FILES_ZIP=b"""
P)h>@6aWAK004j$h)Vzf0000000000000L7003-nV_|G%FHlPZ1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMfJFHlPZ
1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMi7FHlPZ1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMi9FHlPZ
1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMl4FHlPZ1QY-O0001h6^Kg!00000000000000A0001NZ)0I>WiMlMFHlPZ
1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMo5FHlPZ1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMo9FHlPZ
1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMrHFHlPZ1QY-O0001e6^Kg!00000000000000A0001NZ)0I>WiMrOFHlPZNo. The script performs tasks like querying and installing system packages, installing a man page, etc.
I have managed to do the latter (install manpagse) at least with pip. system packages are better (in my opinion) handled by the user or the distributions (unless, of course, they are pip dependencies themselves).
For the record, I have taken a look at install.py but the thing is 8000-lines long and it wasn’t obvious, at least to me, exactly what it was trying to do that pip couldn’t…
If I understand you correctly, you are saying in your opinion it’s better to have the program installable by pip–but for that to work, the user needs to first install by hand all the system package requirements (i.e. deb / rpm files) and take care of the other tasks that the install script does?
that is correct, yes.
like right now, i’d feel more confident installing a pip package than running an arbitrary executable. that way i know what i just installed (and how to uninstall it). as things stand, i will probably install from source using setup.py because I am more familiar with that…
i would also argue that using pip follows the principal of least astonishment (POLA) for people used to install python software… that’s the normal way to setup things, and it’s expected that there might be extra stuff to do…
A few points:
- Run the installer with the -i flag so you can confirm each step if you don’t trust the installer. Since you understand python you are most welcome to audit the installer and publish the results of your audit here, if you want.
- A decade of experience doing this tells me that the vast majority of end users have absolutely no desire to mess about manually installing packages. The only reason the install script exists is because most distros have not yet packaged the 0.9 series, and are still stuck on 0.4.11. I don’t have time to package it for each distro. An Appimage was tried some months ago, with most of the work being done by @Carmelo_DrRaw. However it proved highly time consuming and we haven’t had time to continue the work.
Cool! I didn’t know that was possible, thanks!
Since you understand python you are most welcome to audit the installer and publish the results of your audit here, if you want.
So I would argue that it’s easier for me to audit a ~100 line setup.py script than a 8000 install.py. 
Besides, if I would be to audit something, i would audit the rest of the source code, which I would eventually run on a day to day basis, instead of the source code of an installer I would run only once. Furthermore, that would provide a baseline that future package updates could use, to then only review diffs from the baseline.
But I understand where you’re coming from.
- A decade of experience doing this tells me that the vast majority of end users have absolutely no desire to mess about manually installing packages.
I may not be good model for the majority of users, of course. I don’t particularly want to mess with a manual install either, but I don’t mind doing apt-get install $dependency; pip install foo. I mean you fundamentally always have to install something, if only Python itself… For me, pip install seems less “manual” than a custom installer. Oh and in case that matters, I have also have about a decade of experience both writing and packaging Python software, so I didn’t write this just on a whim… 
The only reason the install script exists is because most distros have not yet packaged the 0.9 series, and are still stuck on 0.4.11. I don’t have time to package it for each distro.
Right. I guess what I hear from you is that I should rather work on the Debian package (which I am also doing). It just seems strange to me to deliberately not publish a pip package when it’s so simple to do… It also makes packaging easier (from Debian’s point of view, at least), since there are often tools to directly convert between packaging formats automatically.
Anyways, sorry for the noise, I’ll just focus on the Debian packaging instead.
Thank you. It will be an enormous service if you can bring the Debian package up to date!
Well, I don’t want to take credit off the people that are actually doing the work here. So far, all I’ve done is ping people in #867527. The main blocker right now is that, well… dependencies are missing. 
But that should be easy enough to fix - i’m just giving time to the current maintainer to complete the work before getting involved…
I must still encourage you to package RPD as a PIP package, it will make things easier for a lot of people!
I don’t know if the Debian packager of Rapid Photo Downloader has abandoned the project or not. He’s never responded to my emails. 0.9.0 was released in July, and nothing has happened to the existing Debian package.
Are you aware that you can install the tar.gz using pip? It’s not obvious to me what is added by putting it up on pypi in addition to launchpad.
Yeah, I think people are just busy… The new update required packaging 4 more python packages as dependencies and for some reason this is taking some time… As I mentioned yesterday, there’s a bug report to track all this:
I don’t have much time to deal with this now, but I will certainly fix this in 2018 if no one else does by then.
Are you aware that you can install the tar.gz using pip? It’s not obvious to me what is added by putting it up on pypi in addition to launchpad.
I wasn’t aware of that. So one thing adding it to PyPI would be to remove annoying questions like mine in the future. It also makes it easier to install than finding the .tar.gz, downloading it by hand, finding it on the filesystem, then running pip on it…
I like this dramatic description of the internet. Somehow I imagine turning a handle to get this file (‘download by hand’). 
2 Likes
So I just tested this, and it failed:
$ sudo pip3 install .
Processing /home/anarcat/src/rapid-photo-downloader
Requirement already satisfied: arrow in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: colorlog in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: colour in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: easygui in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: gphoto2 in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: psutil in /usr/local/lib/python3.5/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: pymediainfo in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: pyprind in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: python-dateutil in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: pyxdg in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: pyzmq in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: rawkit in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Requirement already satisfied: sortedcontainers in /usr/lib/python3/dist-packages (from rapid-photo-downloader==0.9.6)
Building wheels for collected packages: rapid-photo-downloader
Running setup.py bdist_wheel for rapid-photo-downloader ... done
Stored in directory: /root/.cache/pip/wheels/4a/e5/29/ab20494b05214ed2b9029e08e070f3f88e28fa0c99994e1473
Successfully built rapid-photo-downloader
Installing collected packages: rapid-photo-downloader
Exception:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/commands/install.py", line 360, in run
prefix=options.prefix_path,
File "/usr/lib/python3/dist-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/usr/lib/python3/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/usr/lib/python3/dist-packages/pip/wheel.py", line 247, in move_wheel_files
prefix=prefix,
File "/usr/lib/python3/dist-packages/pip/locations.py", line 153, in distutils_scheme
i.finalize_options()
File "/usr/share/python-wheels/setuptools-32.3.1-py2.py3-none-any.whl/setuptools/command/install.py", line 38, in finalize_options
orig.install.finalize_options(self)
File "/usr/lib/python3.5/distutils/command/install.py", line 388, in finalize_options
('build_lib', 'build_lib'))
File "/usr/lib/python3.5/distutils/cmd.py", line 286, in set_undefined_options
src_cmd_obj = self.distribution.get_command_obj(src_cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 857, in get_command_obj
self._set_command_options(cmd_obj, options)
File "/usr/lib/python3.5/distutils/dist.py", line 901, in _set_command_options
% (source, command_name, option))
distutils.errors.DistutilsOptionError: error in setup.cfg: command 'build' has no such option 'icons'
I had to do make the following change for the build to complete:
diff --git a/setup.cfg b/setup.cfg
index 77700a4..9359cf7 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -2,10 +2,10 @@
universal = 0
[build]
-i18n = True
-help = False
-icons = True
-pod2man = True
+#i18n = True
+#help = False
+#icons = True
+#pod2man = True
[build_i18n]
domain = rapid-photo-downloader
Maybe I’m not using the right setuptools version? Any idea what’s going on here?
Anyways, after that change, I can, indeed, install with pip install . so that’s great. For some reason, I don’t have that problem when building the Debian package - presumably because it bypasses normal pip install routines…
Also: I see now there are many other non-Python dependencies required, so I understand better why you don’t want to provide a Pip package. It would have nevertheless made my job generally easier though…
I have mostly packaged the missing python dependencies at this point. Those will new to go through the NEW process, which sometimes takes a while, before RPD can be updated in Debian, but hopefully, the next Debian release will ship with an up to date RPD!
Cheers!
1 Like
Sure. You’re installing python library dependencies using the setuptools config file from Rapid Photo Downloader itself, which is incorrect and causes the error you are seeing.
The correct approach is to install the python library dependencies in one step, without using that config file, and then install the application itself while telling pip to install dependencies. Everything works as expected then.
That is the approach the install script takes, see lines 1432-1479: http://bazaar.launchpad.net/~dlynch3/rapid/zeromq_pyqt/view/head:/install.py#L1432
By commenting out the lines in the setup.cfg file, you are inhibiting the creation of the man page, icons, and translations.
Thank you for doing this. The most important thing now is to ensure that Ubuntu 18.04 will include it. I will monitor this as release deadlines get closer. It’s sometimes possible to request sync a package from Debian before an Ubuntu release even if the mass sync into Ubuntu universe from unstable has already happened.
1 Like
I’m not sure I follow. You’re saying that setup.py is failing because there are missing dependencies? Can you be a little more specific? Which dependencies? Normally, such dependenceis would be specified in the setup_requires field.
That should be possible. I need to upload the package now, which was updated to 0.9.7. Will let you know.
When installing using pip, do it exactly as is done in the install script, in two stages:
- install only the Rapid Photo Downloader dependencies that are installable from pypi, using pip
- only when that is done, install Rapid Photo Downloader while telling pip to ignore the dependencies
cd some/dir/where/there/is/no/setup.cfg
pip3 install --user -r requirements.txt
cd rapid-photo-downloader-dir
pip3 install --user --no-deps rapid-photo-downloader-xyz.tar.gz
If you don’t do that, errors will result because the dependencies will setup be using the Rapid Photo Downloader setup.cfg, which is wrong and causes errors. Seems like a pip usability bug to me, but maybe they have their reasons…
The way you attempted to solve it, by commenting out lines in the setup.cfg, means that the setup.py will not build the application icon or man page. Maybe the Debian build system auto-magically builds them in any case? I have no idea.
I believe Debian has its own way of dealing with that stuff, indeed, so don’t worry about that too much.
For the record, I have just uploaded 0.9.7 into Debian, which will trickle down into testing (in 10 days) and Ubuntu (shortly?).