Are Fujifilm X-trans raw files not supported?


(Sebastien Guyader) #1

I compiled LHDR from github today, and since it relies on lbraw, I thought it could work on Fujifilm xtrans raw files. So I tried to load 4 of such files from my Fuji X-T2, the program didn’t complain but… despite the the CPU meter showing around 50% of use by LDHR, I waited many minutes but nothing seemed to happen, the “HDR creation wizard” showed “loading” and the progress bar stayed at 0% (and hitting teh “cancel” button does nothing).
Are xtrans raw files unsupported? If it’s the case, it would be nice to get some message informing the user that their raw files are not supported, and have the program stop there.


(Franco Comida) #2

Which version of LibRaw is LHDR linked to? Latest 0.19 LibRaw 0.19 should support x-trans better. Check whether your camera is supported from Help->Show Supported Cameras. My next task is to improve raw import with 0.19 and LibRaw in general.


(Sebastien Guyader) #4

@fcomida LHDR is linked to LibRaw 0.19.0 installed from the official repository of my distro (Manjaro Linux). Ther Fujifilm X-T2 is listed as supported, but for whatever reason it makes LHDR stall when I try to load X-T2 files.

Edit: my problem is more serious, it fails to load Pentax and Nikon files. I get the followin in the console:

qt5ct: using qt5ct plugin
libpng warning: iCCP: known incorrect sRGB profile
qt5ct: D-Bus global menu: no
qt5ct: D-Bus system tray: no
qt5ct: palette support is disabled
qt.qpa.xcb: QXcbConnection: XCB error: 5 (BadAtom), sequence: 732, resource id: 0, major code: 19 (DeleteProperty), minor code: 0
void HdrWizard::resizeEvent(QResizeEvent*)
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef"
"LoadFile: Loading data for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef"
"LoadFile: Loading data for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef"
"LoadFile: Loading data for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef"
"LoadFile: Cannot load /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef: Error Processing RAW File"
"LoadFile: Average Luminance for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef is 0.00103521"
"LoadFile: Average Luminance for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef is 0.0172535"
HdrCreationManager::loadFilesDone(): Data loaded ... move to                 internal structure!

And a message window pops up with HdrCreationManager::loadFilesDone(): Error loading a file.


(Morgan Hardwood) #5

@sguyader you can attach GDB to the running process, then get a stack backtrace to see where it’s stuck.


(Sebastien Guyader) #6

Here’s a gdb output from just running luminance-hdr:

Starting program: /usr/bin/luminance-hdr 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffdd520700 (LWP 20122)]
qt5ct: using qt5ct plugin
[New Thread 0x7fffd75d4700 (LWP 20123)]
[New Thread 0x7fffcecf0700 (LWP 20124)]
libpng warning: iCCP: known incorrect sRGB profile
qt5ct: D-Bus global menu: no
[New Thread 0x7fffcd836700 (LWP 20125)]
[New Thread 0x7fffcd035700 (LWP 20126)]
[New Thread 0x7fffcc834700 (LWP 20127)]
qt5ct: D-Bus system tray: no
[New Thread 0x7fffbffff700 (LWP 20128)]
qt5ct: palette support is disabled
qt.qpa.xcb: QXcbConnection: XCB error: 5 (BadAtom), sequence: 733, resource id: 0, major code: 19 (DeleteProperty), minor code: 0

Then after trying to load 3 Pentax files:

void HdrWizard::resizeEvent(QResizeEvent*)
[New Thread 0x7fffbedd6700 (LWP 20148)]
[Thread 0x7fffbedd6700 (LWP 20148) exited]
[New Thread 0x7fffbedd6700 (LWP 20149)]
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef"
[New Thread 0x7fffbe5d5700 (LWP 20150)]
[New Thread 0x7fffbddd4700 (LWP 20151)]
[New Thread 0x7fffbd5d3700 (LWP 20152)]
"LoadFile: Loading data for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef"
"LoadFile: Loading data for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef"
"LoadFile: Loading data for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef"
[New Thread 0x7fffbcb62700 (LWP 20153)]
[New Thread 0x7fff9f145700 (LWP 20154)]
[New Thread 0x7fff9e944700 (LWP 20155)]
[New Thread 0x7fff9cdd2700 (LWP 20156)]
[New Thread 0x7fff9c5d1700 (LWP 20157)]
[New Thread 0x7fff9bdd0700 (LWP 20158)]
[New Thread 0x7fff922d2700 (LWP 20159)]
[New Thread 0x7fff91ad1700 (LWP 20160)]
[New Thread 0x7fff912d0700 (LWP 20161)]
"LoadFile: Cannot load /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_2.pef: Error Processing RAW File"
"LoadFile: Cannot load /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_1.pef: Error Processing RAW File"
"LoadFile: Average Luminance for /home/sguyader/Pictures/test/HDRMerge/amsterdam_moving_boat_3.pef is 0.0172535"
HdrCreationManager::loadFilesDone(): Data loaded ... move to                 internal structure!

(Sebastien Guyader) #7

And here’s the full gdb output from where LHDR is stuck trying to load X-T2 files:

Starting program: /usr/bin/luminance-hdr 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
[New Thread 0x7fffdd520700 (LWP 22801)]
qt5ct: using qt5ct plugin
[New Thread 0x7fffd75d4700 (LWP 22802)]
[New Thread 0x7fffcecf0700 (LWP 22803)]
libpng warning: iCCP: known incorrect sRGB profile
qt5ct: D-Bus global menu: no
[New Thread 0x7fffcd836700 (LWP 22804)]
[New Thread 0x7fffcd035700 (LWP 22805)]
[New Thread 0x7fffcc834700 (LWP 22806)]
qt5ct: D-Bus system tray: no
[New Thread 0x7fffbffff700 (LWP 22807)]
qt5ct: palette support is disabled
qt.qpa.xcb: QXcbConnection: XCB error: 5 (BadAtom), sequence: 736, resource id: 0, major code: 19 (DeleteProperty), minor code: 0
void HdrWizard::resizeEvent(QResizeEvent*)
[New Thread 0x7fffbedd6700 (LWP 22810)]
[Thread 0x7fffbedd6700 (LWP 22810) exited]
[New Thread 0x7fffbedd6700 (LWP 22815)]
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1290.RAF"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1290.RAF"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1291.RAF"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1291.RAF"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1292.RAF"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1292.RAF"
"HdrCreationManager::loadFiles(): Checking /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1293.RAF"
"HdrCreationManager::loadFiles():                             Schedule loading for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1293.RAF"
[New Thread 0x7fffbe5d5700 (LWP 22816)]
[New Thread 0x7fffbddd4700 (LWP 22817)]
[New Thread 0x7fffbd5d3700 (LWP 22818)]
"LoadFile: Loading data for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1290.RAF"
"LoadFile: Loading data for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1292.RAF"
"LoadFile: Loading data for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1291.RAF"
"LoadFile: Loading data for /home/sguyader/Pictures/Photo/Paris_11+12.08.2017/DSCF1293.RAF"
[New Thread 0x7fffbca92700 (LWP 22819)]
[New Thread 0x7fff97d4a700 (LWP 22821)]
[New Thread 0x7fff9e2a5700 (LWP 22820)]
[New Thread 0x7fff97549700 (LWP 22822)]
[New Thread 0x7fff96d48700 (LWP 22823)]
[New Thread 0x7fff96547700 (LWP 22824)]
[New Thread 0x7fff95d46700 (LWP 22825)]
[New Thread 0x7fff95545700 (LWP 22826)]
[New Thread 0x7fff94d44700 (LWP 22827)]
[New Thread 0x7fff94543700 (LWP 22828)]
[New Thread 0x7fff93d42700 (LWP 22829)]
[New Thread 0x7fff93541700 (LWP 22830)]

(Ingo Weyrich) #8

Is noise reduction in lhdr enabled? If so, disable it.


(Sebastien Guyader) #9

@heckflosse Indeed NR was enabled, but turning it off doesn’t improve the situation.


(Ingo Weyrich) #10

Even pef files don’t load after turning it off?


(Sebastien Guyader) #11

Indeed even PEF’s won’t load. I made a fresh cloning and compiling from github, still the same.


(Franco Comida) #12

LibRaw 0.19 has issues with Luminance HDR. I have other reports of errors or even crashes.
My build on Fedora 29 on a VM behave strangely, it crashes at the very first attempt of reading any raw or tells the raw file failed to being read , but then, afterward it works!!!
As I said my next task will be make LHDR working with LibRaw 0.19


(Sebastien Guyader) #13

@fcomida Do you know if the problems are valid even for the latest beta builds of LibRaw? Otherwise I can try compiling it from the current source code.


(Franco Comida) #14

@sguyader I have no idea. Since you are building LibRaw and LHDR from source why don’t you stick with 0.18 for now?


(Sebastien Guyader) #15

I have other programs that rely on LibRaw, installed from my distro’s official repo. Downgrading to LibRaw 0.18 would mean downgrading those programs too
I was just suggesting trying temporarily to link LHDR again the latest beta of LibRaw to see if it would fix the problem.

I’ll try to see if the appimage package has the same problem.


(Franco Comida) #16

Yeah dependencies are a problem.
That’s crazy: I run LHDR linked to 0,19, read any RAW file --> error!!!
Then I open a HDR (EXR, RGBE) or create a HDR from Jpegs, close the HDR and open a RAW again --> IT WORKS !!!
Can you confirm this?


(Sebastien Guyader) #17

No I can’t confirm, I still can’t load raw files (I tried RAFs and PEFs) even after successfully generating an HDR image from raster images.


(Franco Comida) #18

I am quite sure the issue is somehow related to the removal from 0.19 of GPL2 and GPL3 demosicing algos (AMAZE and others). LHDR build system will detect their absence and deal with that (in the GUI for example). Not sure if the configuration is updated though. Can you check the settings? …while writing this I indeed checked “Luminance HDR.conf” and “user_qual” field (demosicing method) is not updated, Either edit the file or change it from the GUI Settings. This way the first attempt still fails but then it works again :open_mouth:


(Sebastien Guyader) #19

I’m not sure I understand what you ask me to do. In Luminance HDR.conf I have this:

[Raw_Conversion_Options]
TK=6500
aber_0=1
aber_2=1
auto_bright=false
brightness=@Variant(\0\0\0\x87?\x80\0\0)
do_not_use_fuji_rotate=false
four_color_rgb=false
green=@Variant(\0\0\0\x87?\x80\0\0)
highlights=0
level=0
med_passes=0
threshold=@Variant(\0\0\0\x87\x42\xc8\0\0)
use_black=false
use_chroma=false
use_noise=false
use_sat=false
user_black=0
user_qual=3
user_sat=20000
wb_method=1

So, I guess user_qual=3 means AHD demosaicing algo. What should I change it to?


(Sebastien Guyader) #20

I tried to set the demosaicing algo to bilinear, processed an HDR image from EXR, saved the result, and tried this in the same session:

  1. to load 3 PEFs -> failed
  2. to load 3 NEFs -> success
  3. to load again the 3 PEFs -> success
  4. to load 4 RAFs -> success -> saved the output, closed the HDR file
  5. to load again the same RAFs -> failed

Go figure what’s going on… :sweat_smile:


(Franco Comida) #21

For now I only know it isn’t working and must be fixed in the next LHDR release.