How to compile ART?

noteasy2beme · April 26, 2022, 3:54pm

Could you please educate me how to enable mi-malloc when I compile ART?

Did you use any specific option when you compile LibRaw with LibRaw-cmake?

Thanks!

agriggio · April 26, 2022, 4:38pm

Hi,
Just build and install mimalloc, then set ENABLE_MIMALLOC=1 in CMakeCache.txt.
For libraw I think I used the default flags for release mode

noteasy2beme · April 27, 2022, 1:45am

This the information I collected before it crashes.
It looks like to me the crash is related to wavelet.
I’m guessing this issue might be caused by the compiler I use (gcc-11), since your build doesn’t have this problem.

Thread 196 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 880.0xf9c]
0x00007ff60085a37d in rtengine::wavelet_level::_ZN8rtengine13wavelet_levelIfE29SynthesisFilterHaarHorizontalEPKfS3_Pfii._omp_fn.0(void) () at C:/msys64/home/ART/rtengine/cplx_wavelet_level.h:252
252 for(int i = 0; i < skip; i++) {
(gdb) bt
#0 0x00007ff60085a37d in rtengine::wavelet_level::_ZN8rtengine13wavelet_levelIfE29SynthesisFilterHaarHorizontalEPKfS3_Pfii._omp_fn.0(void) ()
at C:/msys64/home/ART/rtengine/cplx_wavelet_level.h:252
#1 0x00007ffebc69e8d3 in ?? () from C:\msys64\mingw64\bin\libgomp-1.dll
#2 0x00007ffedfa74d53 in ?? () from C:\msys64\mingw64\bin\libwinpthread-1.dll
#3 0x00007ffee64daf5a in msvcrt!_beginthreadex ()
from C:\WINDOWS\System32\msvcrt.dll
#4 0x00007ffee64db02c in msvcrt!_endthreadex ()
from C:\WINDOWS\System32\msvcrt.dll
#5 0x00007ffee65e7034 in KERNEL32!BaseThreadInitThunk ()
from C:\WINDOWS\System32\kernel32.dll
#6 0x00007ffee6722651 in ntdll!RtlUserThreadStart ()
from C:\WINDOWS\SYSTEM32\ntdll.dll
#7 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

guzzisti · April 28, 2022, 4:04pm

Hi Alberto,

sidequestion:
does ART use/support mimalloc >= 2.0?
The last time i tried mimalloc was on 1.4 or something, current version is 2.0.6 and ART doesn’t detect it - if i understand CMakeLists.txt correctly it’s only looking for 1.4 releases.

agriggio · April 28, 2022, 4:24pm

Hmmm, i don’t know to be honest. I think I have 1.6, and for sure I haven’t tried 2.0. I’ll check and let you know

a286 · April 29, 2022, 12:51pm

On the latest version, my local release build on Windows 10 crashes at startup in the same function. I use latest msys (did pacman -Syu before compilation, so it is gcc 11). I don’t use LibRaw. ART window shows up for a moment and then it crashes. Here is stack trace from gdb:

Thread 7 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 12680.0x33a4]
0x00007ff6e3f3b4e8 in rtengine::wavelet_level::SynthesisFilterHaarHorizontal(float const*, float const*, float*, int, int) [clone ._omp_fn.0] ()
(gdb) bt
#0 0x00007ff6e3f3b4e8 in rtengine::wavelet_level::SynthesisFilterHaarHorizontal(float const*, float const*, float*, int, int) [clone ._omp_fn.0] ()
#1 0x00007ff9fa876e89 in ?? ()
from D:\MyDocs_Dev\msys64\mingw64\bin\libgomp-1.dll
#2 0x00007ff6e40f7a9f in void rtengine::wavelet_level::reconstruct_level(float*, float*, float*, float*, float*, float*, int, int, float) ()
#3 0x00007ff6e40fa2f5 in void rtengine::wavelet_decomposition::reconstruct(float*, float) ()
#4 0x00007ff6e3f3d22f in rtengine::ImProcFunctions::localContrast(rtengine::Imagefloat*) ()
#5 0x00007ff6e3e5023c in rtengine::ImProcFunctions::process(rtengine::ImProcFunctions::Pipeline, rtengine::ImProcFunctions::Stage, rtengine::Imagefloat*) ()
#6 0x00007ff6e3f21dad in rtengine::Thumbnail::processImage(rtengine::procparams::ProcParams const&, rtengine::SensorType, int, rtengine::TypeInterpolation, rtengine::FramesMetaData const*, double&, bool, bool) ()
#7 0x00007ff6e3d0075f in Thumbnail::processThumbImage(rtengine::procparams::ProcParams const&, int, double&) ()
#8 0x00007ff6e40caced in ThumbImageUpdater::Impl::processNextJob() ()
#9 0x00007ff6e41130d0 in std::_Function_handler<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> (), std::__future_base::_Task_setter<std::unique_ptr<std::__future_base::_Result, std::__future_base::_Result_base::_Deleter>, std::__future_base::_Task_state<std::_Bind<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl> ()>, std::allocator, void ()>::_M_run()::{lambda()#1}, void> >::_M_invoke(std::_Any_data const&) ()
#10 0x00007ff6e4111d4b in std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool) ()
#11 0x00007ffa10e154cd in ?? ()
from D:\MyDocs_Dev\msys64\mingw64\bin\libwinpthread-1.dll
#12 0x00007ff6e4150704 in void std::call_once<void (std::__future_base::_State_baseV2::)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool*), std::__future_base::_State_baseV2*, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool>(std::once_flag&, void (std::__future_base::_State_baseV2::&&)(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>, bool*), std::__future_base::_State_baseV2*&&, std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>&&, bool&&) ()
#13 0x00007ff6e4115b08 in std::_Function_handler<void (), rtengine::ThreadPool::enqueue<sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&>(rtengine::ThreadPool::Priority, sigc::bound_mem_functor0<void, ThumbImageUpdater::Impl>&)::{lambda()#1}>::_M_invoke(std::_Any_data const&) ()
#14 0x00007ff6e418a0ad in rtengine::ThreadPool::ThreadPool(unsigned long long)::{lambda()#1}::operator()() const ()
#15 0x00007ff9f8691661 in ?? ()
from D:\MyDocs_Dev\msys64\mingw64\bin\libstdc+±6.dll
#16 0x00007ffa10e14d53 in ?? ()
from D:\MyDocs_Dev\msys64\mingw64\bin\libwinpthread-1.dll
#17 0x000001f45560b04a in msvcrt!_beginthreadex ()
from C:\WINDOWS\System32\msvcrt.dll
#18 0x000001f45560b11c in msvcrt!_endthreadex ()
from C:\WINDOWS\System32\msvcrt.dll
#19 0x00007ffa3af37bd4 in KERNEL32!BaseThreadInitThunk ()
from C:\WINDOWS\System32\kernel32.dll
#20 0x00007ffa3b4cced1 in ntdll!RtlUserThreadStart ()
from C:\WINDOWS\SYSTEM32\ntdll.dll
#21 0x0000000000000000 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

agriggio · April 29, 2022, 3:05pm

Thanks for the info. Would you be able to try with a debug build?
Anyway, I’ll try to understand what is going on.

noteasy2beme · April 30, 2022, 4:07am

Just for your information, the crush I observed can’t be duplicated with the build created in debug mode. It CAN be duplicated either with build compiled in release or release with debug information mode.

ocroquette · May 4, 2022, 9:49pm

Hi,

I built ART from source because otherwise it would not start on Ubuntu 22.04, but it is quite slow compared to the precompiled version I was using before. Here is how I built it: How to build ART on Ubuntu – ocroquette's technical Blog

Am I doing something wrong? How are the “official” versions built? Are the options documented somewhere?

Thank you!

agriggio · May 5, 2022, 5:14am

Hi,
The flags you use look correct. Here are a couple of differences:

I use mi-malloc rather than tcmalloc as a custom allocator, because it provides more reliable performance. tcmalloc seems to suffer under some memory allocation patterns that are somehow triggered by art (unfortunately I didn’t have the time to investigate further)
I use the latest master version of libraw from github for raw decoding
finally, if your version of exiv2 doesn’t support your camera (for example for canon with cr3 output exiv2 needs to be built with special flags, not sure if the stock one has this enabled), art falls back to exiftool for metadata reading, which might also be a bit slower

HTH

agriggio · May 5, 2022, 9:41am

Hi,

it should work now – thanks for reporting!

guzzisti · May 5, 2022, 12:07pm

Thanks, i’ve reenabled mimalloc for the AUR package built from latest master. Gonna apply this to the release package as well if everything is fine.

noteasy2beme · May 22, 2022, 7:57pm

It looks like to me the crash is partially related to the compiler and occurs when wavelet is used. RT crashes the same way after I upgraded msys2.

noteasy2beme · May 29, 2022, 3:06am

I updated msys2 today. The gcc version is 12.xx and it’s able to compile ART and RT properly. In other words, they don’t crash any more even if wavelet is turned on.

a286 · May 31, 2022, 12:53pm

Thanks a lot for the info! I’ve just tried it - updated msys2 (gcc is now 12.1.0), rebuilt ART, and indeed it doesn’t crash anymore. Great!

tankist02 · October 27, 2022, 5:22am

LibRaw question. I use fully updated Fedora 35. Latest LibRaw built and installed from GitHub:

ibRaw$ ./version.sh
0.21.0-RC1

But when configuring ART I get this:

build$ cmake -DCMAKE_BUILD_TYPE=Release ENABLE_LIBRAW=1 …
– WARNING: gcc 11.3.1 is known to miscompile ART when using --ffp-contract=fast, forcing the option to be off
– CMAKE_BUILD_TYPE: Release
– searching for library exiv2 in /usr/lib64
– result: /usr/lib64/libexiv2.so
– searching for library lensfun in /usr/local/lib64
– result: /usr/local/lib64/liblensfun.so
– Checking for module ‘libraw_r>=0.21’
– Package ‘libraw_r’, required by ‘virtual:world’, not found
– libraw not found
– Configuring done
– Generating done
– Build files have been written to: /data/soft/art/build

I do have libraw_r in the standard place:

/usr/local/lib/libraw_r.so.23.0.0
/usr/local/lib/libraw_r.so.23
/usr/local/lib/libraw_r.so
/usr/local/lib/libraw_r.la
/usr/local/lib/libraw_r.a
/usr/local/lib/pkgconfig/libraw_r.pc

And many thanks for creating ART. It is my favorite raw editor even though I’m familiar with RawTherapee and darktable.

kmilos · October 27, 2022, 6:33am

Maybe not “standard” on Fedora? The other libs seem to be under /usr/lib64 and /usr/local/lib64…

Have you tried moving it to /usr/local/lib64? (You’ll need to update the contents of the .pc files to match as well.)

Edit: perhaps starting with a clean LibRaw folder and doing autoreconf -ifv before configure would do all of this for you correctly to begin with?

tankist02 · October 27, 2022, 5:24pm

Thank you, kmilos, for trying to help.

I actually ran autoreconf first - without it configure does not exist.

I also copied the files:

$ sudo cp -av /usr/local/lib/libraw* /usr/local/lib64
‘/usr/local/lib/libraw.a’ → ‘/usr/local/lib64/libraw.a’
‘/usr/local/lib/libraw.la’ → ‘/usr/local/lib64/libraw.la’
‘/usr/local/lib/libraw_r.a’ → ‘/usr/local/lib64/libraw_r.a’
‘/usr/local/lib/libraw_r.la’ → ‘/usr/local/lib64/libraw_r.la’
‘/usr/local/lib/libraw_r.so’ → ‘/usr/local/lib64/libraw_r.so’
‘/usr/local/lib/libraw_r.so.23’ → ‘/usr/local/lib64/libraw_r.so.23’
‘/usr/local/lib/libraw_r.so.23.0.0’ → ‘/usr/local/lib64/libraw_r.so.23.0.0’
‘/usr/local/lib/libraw.so’ → ‘/usr/local/lib64/libraw.so’
‘/usr/local/lib/libraw.so.23’ → ‘/usr/local/lib64/libraw.so.23’
‘/usr/local/lib/libraw.so.23.0.0’ → ‘/usr/local/lib64/libraw.so.23.0.0’

Updated the .pc file:

libdir=${exec_prefix}/lib64

Still the same error:

– Checking for module ‘libraw_r>=0.21’
– Package ‘libraw_r’, required by ‘virtual:world’, not found
– libraw not found

I tried autoconf again with the flags you suggested, rebuilt and reinstalled LibRaw - still it is not found by ART’s cmake.

I guess I need to delve into how exactly cmake tries to locate this particular module…

agriggio · October 27, 2022, 5:45pm

Try setting PKG_CONFIG_PATH to the directory where the libraw.pc file got installed

tankist02 · October 27, 2022, 5:59pm

Great, this worked! You are the man, Alberto, many thanks!