Don't report bugs, contribute to bugs hunting

Software
1

What I’m about to write is grounded in darktable environment but might be easily adapted to any free software.

We just published darktable 3.0rc1. That’s the second release candidate of the soft before the stable release planned for Christmas. This one changes a lot of things, improves a lot of things, but breaks some things too, and we are still trying to fix them.

The release candidates are different from development snapshots since the former have to be compiled manually and target mostly tech-savy people, but the latter are provided as pre-built packages anyone can install. [EDIT: @darix pointed that dev snapshots are also pre-built — but far less advertised than RC] Obviously, when an RC is published, a lot more bugs reports come in compared to the usual (which is the point). In addition, we (I) have built up people expectations about new features, so it’s all the more disappointing when things don’t work out of the box.

So, you go report your bugs, and you are right.

But…

We are the developers. That means we can guarantee you that the software works on our computers. Just as much as you can garantee us that is doesn’t on yours. That doesn’t help, does it ? When fuzzy, incomplete bug reports are posted, imagine you gave us a closed black box saying “the thing inside doesn’t work”. What doesn’t work how and what did you expect ?

There are 3 kinds of bugs:

  1. the ones we can reproduce, usually the nicest,
  2. the ones we can’t, where the nightmare begins,
  3. the ones that are working features misunderstood or misused by users, where RTFM is usually enough.

Bug fixing is an experimental and scientific process in which you try to understand what behaves the same and what behaves differently between a working environment (ours) and a faulty one (yours), or between a working use case and a faulty one, to get an intuition about the causes of the misbehaviour. Bug fixing is something you do, most of the time, without direct access to the faulty system, so we rely solely on the info you give us and what we know of your system. As of today, there are more than 300.000 lines of code in darktable only for the core, that means at least 300.000 possible origins of bugs. And, make no mistake, not a single developer knows every one of these 300.000 bad guys.

Since we usually don’t have access to the failing machines, all we can do is trying to find patterns of reproduction to get a smell of the origin of your bug and narrow down the possible origins of the bug in the code. Then, it’s a silly game of trial and error until we have found the causes. Finally, we still need to fix it without breaking something else, which is never guaranteed.

Usually, these bugs are caused by different GPU drivers (AMD and Intel OpenCL have given us much pleasure lately, they don’t behave at all like Nvidia ones), or different OS (GTK differentied support on Windows and Mac, while trying to come clean on the style and themes, has been a true pleasure too), or different versions of the libraries (supporting GTK 3.18 to 3.24 is nice as well, given their sane habit of deprecating half their API every 2 years).

We are not a big team here. Github lists almost 300 contributors, but the day-to-day cleanup is done essentially by a handful of people, most of them on evenings and weekends aside from their day job, most of them running Linux (with limited debugging possibilities on Win and Mac). As the release is approaching, fatigue increases and it’s not going to cease until December 25th.

So, please, help us help you.

  1. When you report a bug, don’t just dump a backtrace (even if they help), detail what system you use (OS, RAM, CPU, GPU, desktop flavour) but also whether you use OpenCL or not and the version of your driver and of the supported OpenCL set.

  2. If your bug is related to colour, detail what colour profiles you use along your pixel pipeline (working, histogram, display, input) and reset your desktop colour profile to default or sRGB, to see if it helps.

  3. Backup and reset your ~/.config/darktable directory too, and see if that helps. If your problem is with some module, try to move it in the pipe and see if that helps.

  4. Try to send us your raw files + XMP editing histories so we can see exactly what you did to produce the bug.

  5. Explain what you did when things went bad. The actions done prior to the bug are just as useful as the backtrace showing the line of code that failed.

Once you do that, we will get much more information to get started, identify if it’s possibly related to some other bug report, and we will lose less time asking you to provide more details about your config or how you understood the soft was supposed to behave. It feels nicer too.

Thanks and have a happy holiday season !

24 Likes
2

https://www.chiark.greenend.org.uk/~sgtatham/bugs.html

1 Like
3

Thanks for this. I’ve been looking for years for an article that expresses the needs of someone fixing a bug without being as caustic as Eric Raymond.

1 Like
4

also this almost ten year old post still applies:

https://encrypted.pcode.nl/blog/index.html%3Fp=649.html

5

well there is also the user’s and bug finder’s perspective. Everybody has his own truth and everybody’s truth is true. What you are writing is true, I agree but I have reported several bugs that were fixed during the last 2 years or so but in most cases I had to annoy the devs and once even fight with them so they take me seriously. Most bugs that I reported were fixed. But sometimes I think that devs are just lazy or unwilling as far as fixing bugs is concerned and need pressure.

6

In a software company or a project you could prioritize certain tasks and give a bug a higher priority than a new feature.

But here the developers ‘work’ in their free time and nobody ‘has to’ work on a specific feature.

It’s true that users and developers have different views. A user may want new features while a developer wants code refactoring.

In some open source projects I saw a ‘voting’ feature where all users can vote for a new feature or a bug and the developers see what the others want the most. But I don’t see such a feature in Github.

1 Like
7

exactly, this is about wanting to work

8

@pphoto It is not about voting - it is about reporting a problem in a structured and concrete way. That does apply to many situations in life. And that is not easy and requieres often quite some work to isolate the core of the problem.

9

The issue is not “my truth” vs. “your truth”. There is only one truth, and nobody to see it.

The issue is “the problems I’m facing” vs. “the problems you’re facing”. If everybody understands the other’s people problems, we can try to converge toward a common solution.

But if users take devs for wizards and assume they know every bit of the software, and it’s just a matter of time and will to fix any bug, then they don’t understand the devs problems, they just keep adding to them, and at the end they might get ignored because they are not part of the solution.

Of course nobody is willing to look into 10-years-old code written by somebody else who didn’t document it and vanished since then. Especially when it’s intricated obfuscated stuff.

Also, devs all have their specialty: databases, UI & UX, style & design, files I/O, threading, high perf computing, maths & physics…

4 Likes
10

As someone who previously did some work with Intel OpenCL and RT, the information a user should provide for OpenCL issues on Intel is:

  1. What exact Intel CPU model?
  2. Since right now only NEO on Linux is not blacklisted, the further questions are:
    2a) What NEO driver version?
    2b) Is that driver version an official release from Releases · intel/compute-runtime · GitHub ?
    2c) Is that official release not listed as “Production” for the platform provided in 1) ? Is there a version that is listed as “Production” and can the issue be reproduced with that release?
    2d) If not an official release but distribution-packaged, is it reporting version 1.0.0 - and if so, does removing the OpenCL kernel cache fix the issue?
11

You feel you need to make enough noise, enough of a splash, to get an unpaid someone around the world to set aside their family, hobbies, fitness and other activities, possibly even their job, and focus on your problem? I don’t know what bug you’re referring to, but I bet it started with a badly written bug report. Well-written bug reports make developers smile. Yes, the reporter needs to put in some effort, such is life.

9 Likes
12

It’s also important to acknowledge that even well-researched bug-reports following best-practices often aren’t sufficient for a developer to solve the bug. Not because of any mistake by the reporter, but simply because the information missing is specific to both the program and/or the described problem. What I mean to say is that a bug-reporter need not worry about writing the “perfect” bug-report or that they failed if they are asked for additional information. A good bug report makes it easy for the developer to ask specific questions.

3 Likes
13

Bugs often require a back and forth between user and developer, so a well constructed bug report shows the developer that you’re willing to put in the effort to come to a resolution.

6 Likes
14

Yes, yes, yes. I’m an unofficial (and unpaid, of course) first-line support for ImageMagick. I’m reluctant to respond to a woolly report that some feature doesn’t do what the user expects, because I know I’ll have to spend time asking what version of the software, exactly what command they used, what their input files were, what was the output, and what did they expect. It sometimes takes a week to get this basic information, and it’s as painful as pulling teeth.

A user who provides that information up-front will get a more helpful response from me, and probably a faster fix to the problem.

4 Likes
15

Turns out, that’s exactly what happened recently in my life :frowning: (I forgot something really important apparently because of fighting for the bugfixes)

But I think meanwhile I find your reactions funny :wink:

16

I have had good experiences in my 2 bug cases with darktable on MacOS. As stated it was very much a two way street. I reported everything I could then through civil conversation we narrowed it down. One was fixed by next RC just by chance. Was hard to reproduce a random history stack issue. The other was not worth fixing as it turned out to be a OpenCL driver issue which is present in Mojave but not Catalina.

I find the best way is to stay fully involved with the report. As the reporter you can be very valuable because you have the machine with the issue. So follow the wishes of the devs helps keep things moving nicely from my experience. Sure it can get tedious and frustrating but software is complex and a lot of times it really is trial and error especially with hard to reproduce issues and edge cases.

1 Like
17

Hence, this is why there’s a phrase “patch welcome”. It’s a enormous undertaking to develop software as sophisticated as Blender or GIMP. Even Photoflare is a huge task. If programming was much easier, a lot of people would have a software tailored to their need, and many more people would you know, make softwares exactly the way they like and use it. If it was much easier to program, I’d probably make a combination of Krita (insanely flexible NDE) and GIMP (Filters support) and RAW processing softwares (color toning and precise control) by now and get the most out of them.

18

Thanks for this post.

In order to reach the right audience, maybe you could modify this issue template in order to better fit your needs? :slight_smile: