Filmulator crashes after lensfun database update (MacOS Big Sur)

Software Filmulator
1

Hi,
I finally managed to build Filmulator on MacOS Big Sur on an Intel-based iMac.

At first it opened fine, and I could “filmulate” at least one image (looked good BTW) - afterwards I went to the settings and updated the lensfun db.
Now Filmulator reproducably crashes whenever I double-click on a file in the queue. The backtrace below seems to point to lensfun …

Any hints where filmulator stores its lensfun database, and what could go wrong there?

Process: filmulator [42606]
Path: /Applications/Filmulator.app/Contents/MacOS/filmulator
Identifier: com.filmulator
Version: 0.11.1
Code Type: X86-64 (Native)
Parent Process: ??? [1]
Responsible: filmulator [42606]
User ID: 501

Date/Time: 2022-03-27 21:09:02.335 +0200
OS Version: macOS 11.6.5 (20G527)
Report Version: 12
Anonymous UUID: 47921FFC-0DD3-5EAD-6C7C-4A9E95CB1882

Sleep/Wake UUID: 54E80A0E-7687-42A9-87E4-0C91D275AB43

Time Awake Since Boot: 740000 seconds
Time Since Wake: 620000 seconds

System Integrity Protection: disabled

Crashed Thread: 0 Dispatch queue: com.apple.main-thread

Exception Type: EXC_CRASH (SIGABRT)
Exception Codes: 0x0000000000000000, 0x0000000000000000
Exception Note: EXC_CORPSE_NOTIFY

Application Specific Information:
abort() called
filmulator(42606,0x10f878e00) malloc: *** error for object 0x7fd0c52c6700: pointer being freed was not allocated

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 libsystem_kernel.dylib 0x00007fff2066f91e __pthread_kill + 10
1 libsystem_pthread.dylib 0x00007fff2069e5bd pthread_kill + 263
2 libsystem_c.dylib 0x00007fff205f3406 abort + 125
3 libsystem_malloc.dylib 0x00007fff204d3165 malloc_vreport + 548
4 libsystem_malloc.dylib 0x00007fff204d62aa malloc_report + 151
5 liblensfun.2.dylib 0x000000010945f444 lfLens::~lfLens() + 104
6 liblensfun.2.dylib 0x000000010945884c lfDatabase::~lfDatabase() + 152
7 liblensfun.2.dylib 0x000000010945cbc5 lf_db_destroy + 19
8 filmulator 0x000000010856a8fd identifyLens(std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator >) + 2957
9 filmulator 0x0000000108594a5c ParameterManager::selectImage(QString) + 6076
10 filmulator 0x0000000108536e64 ParameterManager::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) + 6900
11 filmulator 0x0000000108539c7f ParameterManager::qt_metacall(QMetaObject::Call, int, void**) + 111
12 org.qt-project.QtQml 0x0000000109be26cb 0x109b02000 + 919243
13 org.qt-project.QtQml 0x0000000109bdf275 0x109b02000 + 905845
14 org.qt-project.QtQml 0x0000000109bdeb0b QV4::QObjectMethod::callInternal(QV4::Value const*, QV4::Value const*, int) const + 1319
15 org.qt-project.QtQml 0x0000000109bf445a 0x109b02000 + 992346
16 org.qt-project.QtQml 0x0000000109bf3345 0x109b02000 + 987973
17 org.qt-project.QtQml 0x0000000109ba5f52 QV4::Function::call(QV4::Value const*, QV4::Value const*, int, QV4::ExecutionContext const*) + 190
18 org.qt-project.QtQml 0x0000000109cdcb59 QQmlJavaScriptExpression::evaluate(QV4::CallData*, bool*) + 547
19 org.qt-project.QtQml 0x0000000109ca30d4 QQmlBoundSignalExpression::evaluate(void**) + 936
20 org.qt-project.QtQml 0x0000000109ca3710 0x109b02000 + 1709840
21 org.qt-project.QtQml 0x0000000109cc6755 QQmlNotifier::emitNotify(QQmlNotifierEndpoint*, void**) + 551
22 org.qt-project.QtCore 0x000000010a0cd779 0x109f7b000 + 1386361
23 org.qt-project.QtQuick 0x0000000108cde97b QQuickMouseArea::mouseDoubleClickEvent(QMouseEvent*) + 323
24 org.qt-project.QtQuick 0x0000000108c8201d QQuickItem::event(QEvent*) + 707
25 org.qt-project.QtWidgets 0x00000001087368a6 QApplicationPrivate::notify_helper(QObject*, QEvent*) + 212
26 org.qt-project.QtWidgets 0x0000000108737702 QApplication::notify(QObject*, QEvent*) + 488
27 org.qt-project.QtCore 0x000000010a0aed7e QCoreApplication::notifyInternal2(QObject*, QEvent*) + 138
28 org.qt-project.QtQuick 0x0000000108c963e5 QQuickWindowPrivate::deliverMouseEvent(QQuickPointerMouseEvent*) + 577
29 org.qt-project.QtQuick 0x0000000108c97d4e QQuickWindowPrivate::deliverPointerEvent(QQuickPointerEvent*) + 220
30 org.qt-project.QtQuick 0x0000000108c98759 QQuickWindowPrivate::handleMouseEvent(QMouseEvent*) + 977
31 org.qt-project.QtGui 0x00000001096f9b2b QWindow::event(QEvent*) + 265
32 org.qt-project.QtQuick 0x0000000108c9473d QQuickWindow::event(QEvent*) + 1125
33 org.qt-project.QtWidgets 0x00000001087368a6 QApplicationPrivate::notify_helper(QObject*, QEvent*) + 212
34 org.qt-project.QtWidgets 0x0000000108737702 QApplication::notify(QObject*, QEvent*) + 488
35 org.qt-project.QtCore 0x000000010a0aed7e QCoreApplication::notifyInternal2(QObject*, QEvent*) + 138
36 org.qt-project.QtGui 0x00000001096ef93f QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) + 2047
37 org.qt-project.QtGui 0x00000001096e1229 QWindowSystemInterface::sendWindowSystemEvents(QFlagsQEventLoop::ProcessEventsFlag) + 91
38 libqcocoa.dylib 0x000000010f26f0c0 0x10f240000 + 192704
39 com.apple.CoreFoundation 0x00007fff2079637c CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION + 17
40 com.apple.CoreFoundation 0x00007fff207962e4 __CFRunLoopDoSource0 + 180
41 com.apple.CoreFoundation 0x00007fff20796064 __CFRunLoopDoSources0 + 242
42 com.apple.CoreFoundation 0x00007fff20794a8c __CFRunLoopRun + 893
43 com.apple.CoreFoundation 0x00007fff2079404c CFRunLoopRunSpecific + 563
44 com.apple.HIToolbox 0x00007fff289dca83 RunCurrentEventLoopInMode + 292
45 com.apple.HIToolbox 0x00007fff289dc6b6 ReceiveNextEventCommon + 284
46 com.apple.HIToolbox 0x00007fff289dc583 _BlockUntilNextEventMatchingListInModeWithFilter + 70
47 com.apple.AppKit 0x00007fff22f9cd72 _DPSNextEvent + 864
48 com.apple.AppKit 0x00007fff22f9b545 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1364
49 com.apple.AppKit 0x00007fff22f8d869 -[NSApplication run] + 586
50 libqcocoa.dylib 0x000000010f26e30a 0x10f240000 + 189194
51 org.qt-project.QtCore 0x000000010a0acbbd QEventLoop::exec(QFlagsQEventLoop::ProcessEventsFlag) + 279
52 org.qt-project.QtCore 0x000000010a0af1df QCoreApplication::exec() + 123
53 filmulator 0x000000010853e1b2 main + 8434
54 libdyld.dylib 0x00007fff206b9f3d start + 1

2

The Lensfun database should be at “~/Library/Application Support/filmulator/version_2” or “/Library/Application Support/filmulator/version_2”.

It looks like there’s just a crash in Lensfun itself; maybe you need to try an older commit or something. Were you building Lensfun from the latest git?

3

I was using the macports 0.3.95 version from macports-ports/graphics/lensfun at 290eb4524628269f885e6cacf0f88a0ffd89c3b3 · macports/macports-ports · GitHub, not building it myself.

Macports installs lensfun 0.3.2 by default, which is incompatible with filmulator. So I installed the newer 0.3.95 following the instructions in howto/InstallingOlderPort – MacPorts … maybe I got something wrong along those steps.

After deleting the lensfun database, filmulator works fine again. Fetching the database through Settings causes it to crash again. Need to think how to debug this, or maybe just start from scratch with installing lensfun anew, building filmulator and see if it still happens.

4

I find that after building, I have two versions of liblensfun.dylib in Filmulator.app/Contents/Frameworks :
liblensfun.dylib
liblensfun.2.dylib
Both are about 285 kB. In a binary compare, they are almost identical, except that liblensfun.dylib contains a path to /opt/local/lib/liblensfun.2.dylib (a symlink) whereas liblensfun.2.dylib contains a path to itself.

Also, the output of the initial configuration step points to another symlink liblensfun.dylib and not the real lib - probably as a result, it does not indicate a specific version (unlike for the other libs like libcurl):
…
– Found LENSFUN: /opt/local/lib/liblensfun.dylib
– Found CURL: /opt/local/lib/libcurl.dylib (found version “7.82.0”)
…

Might this be related to the issue, and how could I make the configuration use the lib instead of the symlink? For some other libs the build instructions request to identify the libs, not symlinks, and how to do that. But not for lensfun …

Any help appreciated, I really would like to get this nice SW to fully work :slight_smile:

5

Unfortunately I don’t really know how linking works on macOS. I really hope you can figure it out (and maybe help me figure out how to get a version that I can distribute!).

6

Finally I got it right - it seems that the Macports version of lensfun is somehow broken, I had to build it myself. Now Filmulator works fine, including lens corrections.
I put the package on my dropbox Dropbox - Filmulator.app - Simplify your life

Thanks again for this nice piece of software, it produces really good results with just a few adjustments (or even none at all).

Do you have any plans to further develop Filmulator, the last commits were a year ago? If so, a few improvements would really add a lot of value:

  • Remember basic GUI settings, like window size and position, size of the queue strip, background color …
  • Rate images in organize view (that would be the natural place for rating IMHO)
  • Shift-selecting multiple images for rating or removal from the queue
  • Is there a way to remove images from the database?
7

I’m glad to hear you like it, that could provide the motivation for me to come back to it (which was previously sapped by spending a few months trying to build the latest LibRaw on Windows).

That’s only in the main branch; TestBuild is where I do my development.

  • Regarding gui settings: I need to look into how to do this.
  • Regarding rating images in organize view: you already can (click on the image so it’s highlighted in green, then move the mouse along the top) but I need to make a better interface for it.
  • Regarding multi-select in the queue: I can do this but it’s not an instant change.
  • There is a way to remove images from the database; you can rate an image to X and then right-click any image in the queue and choose “Forget marked images”.
8

Great to hear that you are still on it … :slight_smile:

How stable is TestBuild in general?

BWT, I built noisereduction today (had to change the cmake configuration and patch some paths in CMakeLists.txt for this), works fine with chroma noise reduction but crashes whenever I set NR Strength > 0. Not critical for me, but is this a known issue and potentially fixed in TestBuild?

9

Testbuild is basically completely stable. It’s my daily driver.

There shouldn’t be any crashing, but maybe it’s out of memory?

10

I have rebuilt from TestBuild, including latest LibRaw snapshot. Uploaded to my dropbox (link above).

Works fine so far, except that it also crashes when I change NR Strength to something > 0. Sounds like a memory issue (actually stack):


VM Regions Near 0x7000035e8ff8:
    MALLOC_NANO (reserved)   600008000000-600020000000 [384.0M] rw-/rwx SM=NUL  reserved VM address space (unallocated)
--> STACK GUARD              7000035e8000-7000035e9000 [    4K] ---/rwx SM=NUL  stack guard for thread 4
    Stack                    7000035e9000-70000366b000 [  520K] rw-/rwx SM=SHM  thread 4

Thread 4 Crashed:: QQuickPixmapReader
0   libsystem_pthread.dylib       	0x00007fff2069a47f ___chkstk_darwin + 55
1   filmulator                    	0x000000010e4c013c ImagePipeline::processImage(ParameterManager*, Interface*, Exiv2::ExifData&, QString, ImagePipeline*) + 30172
2   filmulator                    	0x000000010e53ae05 FilmImageProvider::requestImage(QString const&, QSize*, QSize const&) + 3589
 ....

I have 16 GB installed, and prior to the operation Filmulator uses about 3 GB (with 6 GB free). The image has 20 MP.

11

Oh, it might be that issue again. MacOS has a very small maximum stack size, so it’s not about the total amount of memory but the amount of stack-allocated memory.

I’ll have to revise things so that NR allocates memory on the heap, perhaps.

1 Like
12

By the way, the above link no longer works.

13

Curious, what problem(s) were you having? I’m using the almost latest master pull, compiles with some warnings on gcc 11. Now, I do disable all the --enable stuff, including the samples…

14

I’m trying to build it in msys2, but it’s been a while since I last tried.

Most recently I tried setting up a pkgbuild for it based on msys2’s one but I couldn’t even get the pkgbuild application to run at all.

15

Now, I do a static build for a static link to rawproc, seems to go more straightforwardly. Here’s my …/configure (out-of-tree):

../configure --enable-static --disable-shared --enable-openmp --disable-jasper \
--disable-jpeg --disable-lcms --disable-examples \
PKG_CONFIG_PATH=/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig \ 
--no- create --no-recursion`

FWIW…

16

Does this link work?

17

That link works! I think you’re running into the same packaging issues that I was dealing with- I can’t open that version of filmulator on my mac because we have incompatible lensfun versions installed and the .app package in the link doesn’t have the right one included. We tried running with an undefined behavior sanitizer and an address sanitizer on linux but we didn’t find any issues. Maybe you could try running a debug build and seeing if you still get an error? Even better would be to run a debug build through lldb so that we can know a bit more about the bad access.

18

I guess my issue was different: apparently the format of the (newly) downloaded lens database was not compatible with the older lensfun version that I had installed through Macports. When I ran Filmulator from the command line, error messages were pointing in that direction (something like unexpected elements in the db).
When I locally built the latest version, things were resolved.

I am wondering about your finding about incompatible versions in the app image and what you have installed locally: I thought that just the app-included version is used, and it accesses a db that it downloads itself and that is private to Filmulator - so I would not expect a collision with a locally installed version?

19

I’m glad your issue was resolved! I should clarify. What I mean is that this .app file was compiled with dynamic linkage to the lensfun library. When I try to run this .app file, the dynamic linker in the OS tries to find the lensfun library, but can’t. There is supposed to be a way to put the lensfun library in the .app folder structure and give instructions to the linker on where to find it, but I wasn’t ever able to figure that out completely.

20

That‘s strange that it does not work for you. Actually the lensfun.dylib is contained in the app bundle, under Contents/MacOS/Frameworks, so I assumed the OS would take it from there - and at least on my side it does: I still have an older version of lensfun installed under /opt/local, still it takes the new one.