Odd line with RCD on initial demosaic, disappears if temporarily changing algo

nosle · January 23, 2021, 7:17pm

Running build 8bb06c2c4 developing SOOC DNG’s from a Pentax K-1

My photographs have a 3px black line line running 2px up from the bottom edge of the photograph. The line ends 2px from either vertical edge. See screenshot below.

I noticed in in downsized and exported files but is visible in the editor as seen above. It behaves quite strangely. My default uses RCD/VNG4 demosaic and this is what i see (now portrait oriented photo).

If i change to another RCD algo just to check if touching it helps. But the line remains.

Again

However changing to any non RCD algo makes the line go away.

If I then switch back to RCD the line is still gone

I tried to batch change all images to amaze and then back to rcd but this didn’t remove the line. But doing it in editor works.

On the other hand if I apply neutral profile from the file tab and then immediately reset to default and open a file the line is gone.

The photographs I’m seeing this with all have my kids in them so I won’t upload raws.

Logging this mainly as a heads up for others to check or if anyone else encounters it. It might be any combination of issues creating the problem and I’m yet to fully examine it. I developed the files in a build a few days old, saw the line in the exported files, remembered someone recently having a line bug so pulled and built the most recent and the line is still there even in files I didn’t open before.

nosle · January 23, 2021, 7:29pm

I quickly tested with a dpreview file from Pentax K-1 Sample Image Gallery: Digital Photography Review
And the line persists

Pentax K-1 Sample Image Gallery: Digital Photography Review

This is the full pp3 of the dpreview file

[Version]
AppVersion=5.8-2829-g0e7b39038
Version=349

[Exposure]
Auto=false
Clip=0.02
Compensation=0
Brightness=0
Contrast=0
Saturation=0
Black=0
HighlightCompr=0
HighlightComprThreshold=0
ShadowCompr=50
HistogramMatching=false
CurveFromHistogramMatching=false
ClampOOG=true
CurveMode=FilmLike
CurveMode2=Standard
Curve=0;
Curve2=0;

[HLRecovery]
Enabled=true
Method=Color

[Retinex]
Enabled=false
Str=20
Scal=3
Iter=1
Grad=1
Grads=1
Gam=1.3
Slope=3
Median=false
Neigh=80
Offs=0
Vart=200
Limd=8
highl=4
skal=3
complexMethod=normal
RetinexMethod=high
mapMethod=none
viewMethod=none
Retinexcolorspace=Lab
Gammaretinex=none
CDCurve=0;
MAPCurve=0;
CDHCurve=0;
LHCurve=0;
Highlights=0
HighlightTonalWidth=80
Shadows=0
ShadowTonalWidth=80
Radius=40
TransmissionCurve=1;0;0.5;0.34999999999999998;0.34999999999999998;0.59999999999999998;0.75;0.34999999999999998;0.34999999999999998;1;0.5;0.34999999999999998;0.34999999999999998;
GainTransmissionCurve=1;0;0.10000000000000001;0.34999999999999998;0;0.25;0.25;0.34999999999999998;0.34999999999999998;0.69999999999999996;0.25;0.34999999999999998;0.34999999999999998;1;0.10000000000000001;0;0;

[Local Contrast]
Enabled=false
Radius=80
Amount=0.20000000000000001
Darkness=1
Lightness=1

[Channel Mixer]
Enabled=false
Red=1000;0;0;
Green=0;1000;0;
Blue=0;0;1000;

[Black & White]
Enabled=false
Method=Desaturation
Auto=false
ComplementaryColors=true
Setting=RGB-Rel
Filter=None
MixerRed=33
MixerOrange=33
MixerYellow=33
MixerGreen=33
MixerCyan=33
MixerBlue=33
MixerMagenta=33
MixerPurple=33
GammaRed=0
GammaGreen=0
GammaBlue=0
Algorithm=SP
LuminanceCurve=0;
BeforeCurveMode=Standard
AfterCurveMode=Standard
BeforeCurve=0;
AfterCurve=0;

[Luminance Curve]
Enabled=false
Brightness=0
Contrast=0
Chromaticity=0
AvoidColorShift=false
RedAndSkinTonesProtection=0
LCredsk=true
LCurve=0;
aCurve=0;
bCurve=0;
ccCurve=0;
chCurve=0;
lhCurve=0;
hhCurve=0;
LcCurve=0;
ClCurve=0;

[Sharpening]
Enabled=false
Contrast=20
Method=rld
Radius=0.5
BlurRadius=0.20000000000000001
Amount=200
Threshold=20;80;2000;1200;
OnlyEdges=false
EdgedetectionRadius=1.8999999999999999
EdgeTolerance=1800
HalocontrolEnabled=false
HalocontrolAmount=85
DeconvRadius=0.75
DeconvAmount=87
DeconvDamping=0
DeconvIterations=30

[Vibrance]
Enabled=false
Pastels=0
Saturated=0
PSThreshold=0;75;
ProtectSkins=false
AvoidColorShift=true
PastSatTog=true
SkinTonesCurve=0;

[SharpenEdge]
Enabled=false
Passes=2
Strength=50
ThreeChannels=false

[SharpenMicro]
Enabled=false
Matrix=false
Strength=20
Contrast=20
Uniformity=5

[White Balance]
Enabled=true
Setting=Camera
Temperature=5299
Green=0.94899999999999995
Equal=1
TemperatureBias=0

[Color appearance]
Enabled=false
Degree=90
AutoDegree=true
Degreeout=90
AutoDegreeout=true
Surround=Average
complex=normal
ModelCat=02
Surrsrc=Average
AdaptLum=16
Badpixsl=0
Model=RawT
Illum=i50
Algorithm=JC
J-Light=0
Q-Bright=0
C-Chroma=0
S-Chroma=0
M-Chroma=0
J-Contrast=0
Q-Contrast=0
H-Hue=0
RSTProtection=0
AdaptScene=2000
AutoAdapscen=true
YbScene=18
Autoybscen=true
SurrSource=false
Gamut=true
Tempout=5003
Autotempout=true
Greenout=1
Tempsc=5003
Greensc=1
Ybout=18
Datacie=false
Tonecie=false
Presetcat02=false
CurveMode=Lightness
CurveMode2=Lightness
CurveMode3=Chroma
Curve=0;
Curve2=0;
Curve3=0;

[Impulse Denoising]
Enabled=false
Threshold=50

[Defringing]
Enabled=false
Radius=2
Threshold=13
HueCurve=1;0.16666666699999999;0;0.34999999999999998;0.34999999999999998;0.34699999999999998;0;0.34999999999999998;0.34999999999999998;0.51366742600000004;0;0.34999999999999998;0.34999999999999998;0.66894457100000004;0;0.34999999999999998;0.34999999999999998;0.82877752459999998;0.97835991;0.34999999999999998;0.34999999999999998;0.99088838270000001;0;0.34999999999999998;0.34999999999999998;

[Dehaze]
Enabled=false
Strength=50
ShowDepthMap=false
Depth=25
Saturation=50

[Directional Pyramid Denoising]
Enabled=false
Enhance=false
Median=false
Luma=0
Ldetail=0
Chroma=15
Method=Lab
LMethod=SLI
CMethod=MAN
C2Method=AUTO
SMethod=shal
MedMethod=soft
RGBMethod=soft
MethodMed=Lonly
Redchro=0
Bluechro=0
Gamma=1.7
Passes=1
LCurve=1;0.050000000000000003;0.14999999999999999;0.34999999999999998;0.34999999999999998;0.55000000000000004;0.040000000000000001;0.34999999999999998;0.34999999999999998;
CCCurve=1;0.050000000000000003;0.5;0.34999999999999998;0.34999999999999998;0.34999999999999998;0.050000000000000003;0.34999999999999998;0.34999999999999998;

[EPD]
Enabled=false
Strength=0.5
Gamma=1
EdgeStopping=1.3999999999999999
Scale=1
ReweightingIterates=0

[FattalToneMapping]
Enabled=false
Threshold=30
Amount=20
Anchor=50

[Shadows & Highlights]
Enabled=false
Highlights=0
HighlightTonalWidth=70
Shadows=0
ShadowTonalWidth=30
Radius=40
Lab=false

[Crop]
Enabled=false
X=0
Y=0
W=4924
H=7372
FixedRatio=true
Ratio=4:3
Orientation=As Image
Guide=Frame

[Coarse Transformation]
Rotate=0
HorizontalFlip=false
VerticalFlip=false

[Common Properties for Transformations]
Method=log
AutoFill=true

[Rotation]
Degree=0

[Distortion]
Amount=0

[LensProfile]
LcMode=none
LCPFile=
UseDistortion=true
UseVignette=true
UseCA=false
LFCameraMake=
LFCameraModel=
LFLens=

[Perspective]
Method=camera_based
Horizontal=0
Vertical=0
CameraCropFactor=1
CameraFocalLength=24
CameraPitch=0
CameraRoll=0
CameraShiftHorizontal=0
CameraShiftVertical=0
CameraYaw=0
ProjectionShiftHorizontal=0
ProjectionPitch=0
ProjectionRotate=0
ProjectionShiftVertical=0
ProjectionYaw=0
ControlLineValues=
ControlLineTypes=

[Gradient]
Enabled=false
Degree=0
Feather=25
Strength=0.59999999999999998
CenterX=0
CenterY=0

[Locallab]
Enabled=false
Selspot=0

[PCVignette]
Enabled=false
Strength=0.59999999999999998
Feather=50
Roundness=50

[CACorrection]
Red=0
Blue=0

[Vignetting Correction]
Amount=0
Radius=50
Strength=1
CenterX=0
CenterY=0

[Resize]
Enabled=false
Scale=0.28000000000000003
AppliesTo=Cropped area
Method=Lanczos
DataSpecified=3
Width=2048
Height=2048
AllowUpscaling=false

[PostDemosaicSharpening]
Enabled=false
Contrast=6
AutoContrast=true
AutoRadius=true
DeconvRadius=0.82999999999999996
DeconvRadiusOffset=0
DeconvIterCheck=true
DeconvIterations=20

[PostResizeSharpening]
Enabled=true
Contrast=0
Method=rld
Radius=0.5
Amount=200
Threshold=20;80;2000;1200;
OnlyEdges=false
EdgedetectionRadius=1.8999999999999999
EdgeTolerance=1800
HalocontrolEnabled=false
HalocontrolAmount=85
DeconvRadius=0.45000000000000001
DeconvAmount=23
DeconvDamping=0
DeconvIterations=100

[Color Management]
InputProfile=(cameraICC)
ToneCurve=false
ApplyLookTable=false
ApplyBaselineExposureOffset=false
ApplyHueSatMap=true
DCPIlluminant=0
WorkingProfile=ProPhoto
WorkingTRC=none
WorkingTRCGamma=2.3999999999999999
WorkingTRCSlope=12.92
OutputProfile=RTv4_sRGB
OutputProfileIntent=Relative
OutputBPC=true

[Wavelet]
Enabled=false
Strength=100
Balance=0
Sigmafin=1
Sigmaton=1
Sigmacol=1
Sigmadir=1
Rangeab=20
Protab=0
Iter=0
MaxLev=7
TilesMethod=full
complexMethod=normal
denMethod=12low
mixMethod=mix7
sliMethod=sli
quaMethod=cons
DaubMethod=4_
ChoiceLevMethod=all
BackMethod=grey
LevMethod=4
DirMethod=all
CBgreenhigh=0
CBgreenmed=0
CBgreenlow=0
CBbluehigh=0
CBbluemed=0
CBbluelow=0
Ballum=7
Sigm=1
Levden=0
Thrden=0
Limden=0
Balchrom=0
Chromfine=0
Chromcoarse=0
MergeL=20
MergeC=20
Softrad=0
Softradend=0
Strend=50
Detend=0
Thrend=0
Expcontrast=false
Expchroma=false
Expedge=false
expbl=false
Expresid=false
Expfinal=false
Exptoning=false
Expnoise=false
Expclari=false
LabGridALow=0
LabGridBLow=0
LabGridAHigh=0
LabGridBHigh=0
Contrast1=0
Contrast2=0
Contrast3=0
Contrast4=0
Contrast5=0
Contrast6=0
Contrast7=0
Contrast8=0
Contrast9=0
Chroma1=0
Chroma2=0
Chroma3=0
Chroma4=0
Chroma5=0
Chroma6=0
Chroma7=0
Chroma8=0
Chroma9=0
ContExtra=0
HSMethod=with
HLRange=50;75;100;98;
SHRange=0;2;50;25;
Edgcont=0;10;75;40;
Level0noise=0;0;
Level1noise=0;0;
Level2noise=0;0;
Level3noise=0;0;
Leveldenoise=0;0;
Levelsigm=1;1;
ThresholdHighlight=4
ThresholdShadow=5
Edgedetect=90
Edgedetectthr=20
EdgedetectthrHi=0
Edgesensi=60
Edgeampli=10
ThresholdChroma=5
CHromaMethod=link
Medgreinf=less
Ushamethod=clari
CHSLromaMethod=SL
EDMethod=CU
NPMethod=none
BAMethod=none
TMMethod=cont
ChromaLink=0
ContrastCurve=1;0;0.25;0.34999999999999998;0.34999999999999998;0.5;0.75;0.34999999999999998;0.34999999999999998;0.90000000000000002;0;0.34999999999999998;0.34999999999999998;
blcurve=1;0;0;0;0.34999999999999998;0.5;0;0.34999999999999998;0.34999999999999998;1;0;0.34999999999999998;0.34999999999999998;
Pastlev=0;2;30;20;
Satlev=30;45;130;100;
OpacityCurveRG=1;0;0.5;0.34999999999999998;0.34999999999999998;1;0.5;0.34999999999999998;0.34999999999999998;
Levalshc=1;0;1;0.34999999999999998;0.34999999999999998;0.14999999999999999;0.90000000000000002;0.34999999999999998;0.34999999999999998;0.40000000000000002;0.80000000000000004;0.34999999999999998;0.34999999999999998;0.40000000000000002;0.5;0.34999999999999998;0.34999999999999998;0.5;0.5;0.34999999999999998;0.34999999999999998;0.5;0.20000000000000001;0.34999999999999998;0.34999999999999998;0.80000000000000004;0.10000000000000001;0.34999999999999998;0.34999999999999998;1;0;0.34999999999999998;0.34999999999999998;
OpacityCurveBY=1;0;0.5;0.34999999999999998;0.34999999999999998;1;0.5;0.34999999999999998;0.34999999999999998;
wavdenoise=1;0;1;0.34999999999999998;0.34999999999999998;0.5;1;0.34999999999999998;0.34999999999999998;1;1;0.34999999999999998;0.34999999999999998;
wavdenoiseh=1;0;1;0.34999999999999998;0.34999999999999998;0.5;1;0.34999999999999998;0.34999999999999998;1;1;0.34999999999999998;0.34999999999999998;
OpacityCurveW=1;0;0.34999999999999998;0.34999999999999998;0;0.34999999999999998;0.75;0.34999999999999998;0.34999999999999998;0.59999999999999998;0.75;0.34999999999999998;0.34999999999999998;1;0.34999999999999998;0;0;
OpacityCurveWL=1;0;0.5;0.34999999999999998;0.34999999999999998;1;0.5;0.34999999999999998;0.34999999999999998;
HHcurve=0;
Wavguidcurve=0;
Wavhuecurve=0;
CHcurve=0;
WavclCurve=0;
Median=false
Medianlev=false
Linkedg=false
CBenab=false
Lipst=false
Skinprotect=0
chrwav=0
bluwav=1
Hueskin=-5;25;170;120;
Edgrad=15
Edgeffect=1
Edgval=0
ThrEdg=10
AvoidColorShift=false
Showmask=false
Oldsh=true
TMr=false
Sigma=1
Offset=1
Lowthr=40
ResidualcontShadow=0
ResidualcontHighlight=0
ThresholdResidShadow=30
ThresholdResidHighLight=70
Residualradius=40
Residualchroma=0
Residualblur=0
Residualblurc=0
ResidualTM=0
ResidualEDGS=1.3999999999999999
ResidualSCALE=1
Residualgamma=1
HueRangeResidual=0
HueRange=-260;-250;-130;-140;
Contrast=0

[Directional Pyramid Equalizer]
Enabled=false
Gamutlab=false
cbdlMethod=bef
Mult0=1
Mult1=1
Mult2=1
Mult3=1
Mult4=1
Mult5=1
Threshold=0.20000000000000001
Skinprotect=0
Hueskin=-5;25;170;120;

[HSV Equalizer]
Enabled=false
HCurve=0;
SCurve=0;
VCurve=0;

[SoftLight]
Enabled=false
Strength=30

[Film Simulation]
Enabled=false
ClutFilename=
Strength=100

[RGB Curves]
Enabled=false
LumaMode=false
rCurve=0;
gCurve=0;
bCurve=0;

[ColorToning]
Enabled=false
Method=LabRegions
Lumamode=true
Twocolor=Std
Redlow=0
Greenlow=0
Bluelow=0
Satlow=0
Balance=0
Sathigh=0
Redmed=0
Greenmed=0
Bluemed=0
Redhigh=0
Greenhigh=0
Bluehigh=0
Autosat=true
OpacityCurve=1;0;0.29999999999999999;0.34999999999999998;0;0.25;0.80000000000000004;0.34999999999999998;0.34999999999999998;0.69999999999999996;0.80000000000000004;0.34999999999999998;0.34999999999999998;1;0.29999999999999999;0;0;
ColorCurve=1;0.050000000000000003;0.62;0.25;0.25;0.58499999999999996;0.11;0.25;0.25;
SatProtectionThreshold=30
SaturatedOpacity=80
Strength=50
HighlightsColorSaturation=60;80;
ShadowsColorSaturation=80;208;
ClCurve=3;0;0;0.34999999999999998;0.65000000000000002;1;1;
Cl2Curve=3;0;0;0.34999999999999998;0.65000000000000002;1;1;
LabGridALow=0
LabGridBLow=0
LabGridAHigh=0
LabGridBHigh=0
LabRegionA_1=0
LabRegionB_1=0
LabRegionSaturation_1=0
LabRegionSlope_1=1
LabRegionOffset_1=0
LabRegionPower_1=1
LabRegionHueMask_1=1;0.16666666699999999;1;0.34999999999999998;0.34999999999999998;0.82877752459999998;1;0.34999999999999998;0.34999999999999998;
LabRegionChromaticityMask_1=1;0;1;0.34999999999999998;0.34999999999999998;1;1;0.34999999999999998;0.34999999999999998;
LabRegionLightnessMask_1=1;0;1;0.34999999999999998;0.34999999999999998;1;1;0.34999999999999998;0.34999999999999998;
LabRegionMaskBlur_1=0
LabRegionChannel_1=-1
LabRegionsShowMask=-1

[RAW]
DarkFrame=/szeva
DarkFrameAuto=false
FlatFieldFile=/szeva
FlatFieldAutoSelect=false
FlatFieldBlurRadius=32
FlatFieldBlurType=Area Flatfield
FlatFieldAutoClipControl=false
FlatFieldClipControl=0
CA=false
CAAvoidColourshift=false
CAAutoIterations=2
CARed=0
CABlue=0
HotPixelFilter=false
DeadPixelFilter=false
HotDeadPixelThresh=100
PreExposure=1

[RAW Bayer]
Method=rcdvng4
Border=4
ImageNum=1
CcSteps=0
PreBlack0=0
PreBlack1=0
PreBlack2=0
PreBlack3=0
PreTwoGreen=true
LineDenoise=0
LineDenoiseDirection=3
GreenEqThreshold=0
DCBIterations=2
DCBEnhance=true
LMMSEIterations=2
DualDemosaicAutoContrast=true
DualDemosaicContrast=7
PixelShiftMotionCorrectionMethod=1
PixelShiftEperIso=0
PixelShiftSigma=1
PixelShiftShowMotion=false
PixelShiftShowMotionMaskOnly=false
pixelShiftHoleFill=true
pixelShiftMedian=false
pixelShiftGreen=true
pixelShiftBlur=true
pixelShiftSmoothFactor=0.69999999999999996
pixelShiftEqualBright=false
pixelShiftEqualBrightChannel=false
pixelShiftNonGreenCross=true
pixelShiftDemosaicMethod=amaze
PDAFLinesFilter=false

[RAW X-Trans]
Method=3-pass (best)
DualDemosaicAutoContrast=true
DualDemosaicContrast=20
Border=7
CcSteps=0
PreBlackRed=0
PreBlackGreen=0
PreBlackBlue=0

[MetaData]
Mode=0

[Film Negative]
Enabled=false
RedRatio=1.3600000000000001
GreenExponent=1.5
BlueRatio=0.85999999999999999
ColorSpace=1
RefInput=0;0;0;
RefOutput=2739.947021484375;2740.07568359375;2740.035400390625;
BackCompat=0

[RAW Preprocess WB]
Mode=1

heckflosse · January 23, 2021, 7:49pm

I will have a look tomorrow.

nosle · January 23, 2021, 8:08pm

Ok new test using another camera and another dynamic pp3. Used this image Pentax K-1 II sample gallery: Digital Photography Review
from a different Pentax camera (similar model) and used this minimal pp3.
[Version]
AppVersion=5.8-2829-g0e7b39038
Version=349

[RAW Bayer]
Method=rcdvng4
DualDemosaicAutoContrast=true
DualDemosaicContrast=6

[RAW X-Trans]
Method=3-pass (best)
DualDemosaicAutoContrast=true
DualDemosaicContrast=20

[Film Negative]
BackCompat=0

Still the line

When testing the above minimal pp3 as a dynamic profile for Leica SL2-S and NIkon D780 files I can’t produce the line. So may be a Pentax K-1 XX issue.

The strangest thing is that it disappears by fully demosaicing using a different algo then going back to using RCD. So to see it you have to have RCD as a dynamic profile.

heckflosse · January 23, 2021, 9:55pm

@nosle Works fine here

nosle · January 23, 2021, 11:06pm

Did you set RCD with a dynamic profile? It seems it doesn’t happen if you apply RCD after first using a different demosaicer.

I will have to try cleaning out my settings and testing fresh. Perhaps there’s some config or pp3 interfering.

nosle · January 24, 2021, 9:49am

So i started from scratch on another computer (Debian testing, workstation is sid). renamed config directories. Used the build-rawtherapee script. Assigned the mini pp3 as default raw profile instead of dynamic just to test. I’m using dpreview files from Pentax K-1 and K-1 Mark II sample galleries.

Result the line is still there and it can be removed by switching demosaicer and back again.

With the default set to a RCD only pp3 (no dual demosaic) the problem still remains.

heckflosse · January 24, 2021, 10:25am

I can reproduce it. Fix follows soon. Thanks for reporting

heckflosse · January 24, 2021, 10:28am

I pushed a fix.

nosle · January 24, 2021, 10:36am

The fix seems to solve the problem! Can’t provoke it with the usual tests.

hannoschwalm · January 24, 2021, 12:26pm

@heckflosse and @nosle. Ingo made me aware of this discussion, i just tested on dt with RCD and RCD+VNG4 on the referred pentax images. On dt there seems to be no issue…

There are subtle difference between the implementations but rt takes the image borders in a different way for dng files. The tags in the file header say 7360x4912 while rt uses 7368x4924. I know it is fine for most files but … I checked with “corrected” bayer cropping as rt does and still can’t reproduce though.