Keep the Raws Coming


(system) #1

Keep the Raws Coming

Moar samples!

Our friendly neighborhood @LebedevRI pointed out to me a little while ago that we had reached some nice milestones for https://raw.pixls.us. Not surprisingly I had spaced out and not written anything about it (or really any sort of social posts). Bad Pat!

So let’s talk about raw.pixls.us (RPU) a bit!

Recap

For anyone not familiar with RPU, a quick recap (we had previously written about raw.pixls.us earlier this year). There used to be a website for housing a repository of raw files for as many digital cameras as possible called rawsamples.ch. It was created by Jakob Rohrbach and had been running since March of 2007. Back in 2016 the site was hit with a SQL injection attack that left the Joomla database corrupted (in a teachable moment, the site also didn’t have a database backup).

With the rawsamples.ch site down, @LebedevRI and @andabata worked to get a replacement option in-place and working: https://raw.pixls.us!

Sexy Stats

We grabbed all the files we could salvage from rawsamples.ch and @andabata setup the new page. We’ve had a slowly growing response as folks have filled in gaps for camera models we still don’t have.

For reference, we currently have count of unique cameras in the archive unique cameras, and total count of unique samples unique samples.

Moar samples!

As @LebedevRI has said, we still really need folks to check RPU and send us more samples!

  • We currently only have about 77% coverage.
  • We want to replace any non-CC0 (public domain) samples with CC0 licensed samples.
  • We are still missing some rarer samples like any medium-format or Sigma samples.

Our hope is that some casual reader out there might look at the list and say “Hey! I’ve got that camera lying around - let me submit a sample!”.

Here’s the current list of missing camera samples:

Canon EOS Kiss Digital F
Canon EOS Kiss X7
Canon EOS Kiss X70
Canon EOS Kiss X80
Canon EOS Kiss X9
Canon EOS Rebel SL2
Canon EOS Kiss Digital
Canon EOS Kiss Digital X
Canon Kiss Digital X2
Canon Kiss X2
Canon EOS 5DS
Canon EOS Kiss X5
Canon EOS Kiss X6i
Canon EOS Rebel T4i
Canon EOS Kiss X7i
Canon EOS Kiss X8i
Canon EOS 8000D
Canon EOS Rebel T6s
Canon EOS 9000D
Canon EOS Kiss X9i
Canon EOS M10
Canon EOS M2
Canon PowerShot G9 X
Canon PowerShot S95
Canon PowerShot SX260 HS
Fujifilm FinePix HS30EXR
Fujifilm FinePix HS50EXR
Fujifilm FinePix S100FS
Fujifilm FinePix S5200
Fujifilm FinePix S5500
Fujifilm FinePix S6000fd
Fujifilm FinePix S9000
Fujifilm FinePix S9600fd
Fujifilm IS-1
Fujifilm XF1
Fujifilm XQ2
Kodak EasyShare Z980
Kodak P880
Leaf Aptus-II 5
Leaf Credo 40
Leaf Credo 60
Leaf Credo 80
Leica D-LUX 4
Leica D-LUX 5
Leica D-LUX 6
Leica X2
Minolta DiMAGE 5
Minolta Alpha 5D
Minolta Maxxum 5D
Minolta Alpha 7D
Minolta Maxxum 7D
Nikon 1 J3
Nikon 1 J4
Nikon 1 S1
Nikon 1 V3
Nikon Coolpix A
Nikon Coolpix P7700
Nikon D1H
Nikon D2H
Nikon D2Hs
Nikon D3S
Nikon D4S
Nokia Lumia 1020
Olympus E-10
Olympus E-400
Olympus E-PL1
Olympus E-PL2
Olympus SP320
Olympus SP570UZ
Olympus Stylus1
Olympus XZ-10
Panasonic DMC-FZ80
Panasonic DMC-FZ85
Panasonic DC-FZ91
Panasonic DC-FZ92
Panasonic DC-FZ93
Panasonic DC-ZS70
Panasonic DMC-FX150
Panasonic DMC-FZ100
Panasonic DMC-FZ35
Panasonic DMC-FZ40
Panasonic DMC-FZ50
Panasonic DMC-G5
Panasonic DMC-G8
Panasonic DMC-G85
Panasonic DMC-GF2
Panasonic DMC-GM5
Panasonic DMC-LX9
Panasonic DMC-TZ110
Panasonic DMC-ZS110
Panasonic DMC-ZS40
Panasonic DMC-ZS50
Panasonic DMC-TZ85
Panasonic DMC-ZS60
Pentax 645Z
Pentax K2000
Pentax Q10
Pentax Q7
Phase One IQ250
Ricoh GR
Ricoh GR II
Samsung EK-GN120
Samsung GX10
Samsung GX20
Samsung NX10
Samsung NX1000
Samsung NX11
Samsung NX1100
Samsung NX20
Samsung NX2000
Samsung NX210
Samsung NX5
Sinar Hy6
Sony DSC-RX1
Sony DSC-RX1R
Sony DSLR-A230
Sony DSLR-A290
Sony DSLR-A380
Sony DSLR-A390
Sony DSLR-A450
Sony DSLR-A500
Sony DSLR-A560
Sony ILCE-3000
Sony ILCE-3500
Sony NEX-5N
Sony NEX-C3
Sony NEX-F3
Sony SLT-A33

If you have any of the cameras on this list and don’t mind spending a few minutes uploading a sample file, we would be very grateful for the help!

Don’t forget that we are looking for:

  • Lens mounted on the camera, cap off
  • Image in focus and properly exposed
  • Landscape orientation

and we are not looking for:

  • Series of images with different ISO, aperture, shutter, wb, lighting, or different lenses
  • DNG files created with Adobe DNG Converter
  • Photographs of people, for legal reasons.

If you don’t see your camera on this list, you’re not off the hook yet! We are also looking for files that are licensed very freely…

Non Creative-Commons Zero (CC0)

We have many raw samples that were not licensed as freely as we would like. Ideally we are looking for images that have been released Creative Commons Zero (CC0). This list is all samples we already have that are not licensed CC0, so if you happen to have one of the cameras listed below please consider uploading some new samples for us!

Canon IXUS900Ti
Canon PowerShot A550
Canon PowerShot A570 IS
Canon PowerShot A610
Canon PowerShot A620
Canon PowerShot A630
Canon Powershot A650
Canon PowerShot A710 IS
Canon PowerShot G7
Canon PowerShot S2 IS
Canon PowerShot S5 IS
Canon PowerShot SD750
Canon Powershot SX110IS
Canon EOS 10D
Canon EOS 1200D
Canon EOS-1D
Canon EOS-1D Mark II
Canon EOS-1D Mark III
Canon EOS-1D Mark II N
Canon EOS-1D Mark IV
Canon EOS-1Ds
Canon EOS-1Ds Mark II
Canon EOS-1Ds Mark III
Canon EOS-1D X
Canon EOS 300D
Canon EOS 30D
Canon EOS 400D
Canon EOS 40D
Canon EOS 760D
Canon EOS D2000C
Canon EOS D60
Canon EOS Digital Rebel XS
Canon EOS M
Canon EOS Rebel T3
Canon EOS Rebel T6i
Canon PowerShot A3200 IS
Canon Powershot A720 IS
Canon PowerShot G10
Canon PowerShot G11
Canon PowerShot G12
Canon PowerShot G15
Canon PowerShot G1
Canon PowerShot G1 X Mark II
Canon PowerShot G2
Canon PowerShot G3
Canon PowerShot G5
Canon PowerShot G5 X
Canon PowerShot G6
Canon PowerShot Pro1
Canon PowerShot Pro70
Canon PowerShot S30
Canon PowerShot S40
Canon PowerShot S45
Canon PowerShot S50
Canon PowerShot S60
Canon PowerShot S70
Canon PowerShot S90
Canon PowerShot SD450
Canon Powershot SX110IS
Canon PowerShot SX130 IS
Canon PowerShot SX1 IS
Canon PowerShot SX50 HS
Canon PowerShot SX510 HS
Canon PowerShot SX60 HS
Canon Poweshot S3IS
Epson R-D1
Fujifilm FinePix E550
Fujifilm FinePix E900
Fujifilm FinePix F600EXR
Fujifilm FinePix F700
Fujifilm FinePix F900EXR
Fujifilm FinePix HS10 HS11
Fujifilm FinePix HS20EXR
Fujifilm FinePix S200EXR
Fujifilm FinePix S2Pro
Fujifilm FinePix S3Pro
Fujifilm FinePix S5000
Fujifilm FinePix S5600
Fujifilm FinePix S6500fd
Fujifilm FinePix X100
Fujifilm X100S
Fujifilm X-A2
Fujifilm XQ1
Hasselblad CF132
Hasselblad CFV
Hasselblad H3D
Kodak DC120
Kodak DC50
Kodak DCS460D
Kodak DCS560C
Kodak DCS Pro SLR/n
Kodak EOS DCS 1
Kodak Kodak C330
Kodak Kodak C603 / Kodak C643
Kodak Z1015 IS
Leaf Aptus 75
Leaf Leaf Aptus 22
Leica Leica Digilux 2
Leica Leica D-LUX 3
Leica M8
Leica M (Typ 240)
Leica V-LUX 1
Mamiya ZD
Minolta DiMAGE 7
Minolta DiMAGE 7Hi
Minolta DiMAGE 7i
Minolta DiMAGE A1
Minolta DiMAGE A200
Minolta DiMAGE A2
Minolta Dimage Z2
Minolta Dynax 5D
Minolta Dynax 7D
Minolta RD-175
Minolta RD-175
Nikon 1 S2
Nikon 1 V1
Nikon Coolpix P340
Nikon Coolpix P6000
Nikon Coolpix P7000
Nikon Coolpix P7100
Nikon D100
Nikon D1
Nikon D1X
Nikon D2X
Nikon D300S
Nikon D3
Nikon D3X
Nikon D40
Nikon D60
Nikon D70
Nikon D800
Nikon D80
Nikon D810
Nikon E5400
Nikon E5700
Nikon LS-5000
Nokia Lumia 1020
Olympus C5050Z
Olympus C5060WZ
Olympus C8080WZ
Olympus E-1
Olympus E-20
Olympus E-300
Olympus E-30
Olympus E-330
Olympus E-3
Olympus E-420
Olympus E-450
Olympus E-500
Olympus E-510
Olympus E-520
Olympus E-5
Olympus E-600
Olympus E-P1
Olympus E-P2
Olympus E-P3
Olympus E-PL5
Olympus SP350
Olympus SP500UZ
Olympus XZ-1
Panasonic DMC-FZ150
Panasonic DMC-FZ18
Panasonic DMC-FZ200
Panasonic DMC-FZ28
Panasonic DMC-FZ30
Panasonic DMC-FZ38
Panasonic DMC-FZ70
Panasonic DMC-FZ72
Panasonic DMC-FZ8
Panasonic DMC-G1
Panasonic DMC-G3
Panasonic DMC-GF3
Panasonic DMC-GF5
Panasonic DMC-GF7
Panasonic DMC-GH2
Panasonic DMC-GH3
Panasonic DMC-GH4
Panasonic DMC-GM1
Panasonic DMC-GX7
Panasonic DMC-L10
Panasonic DMC-L1
Panasonic DMC-LF1
Panasonic DMC-LX1
Panasonic DMC-LX2
Panasonic DMC-LX3
Panasonic DMC-LX5
Panasonic DMC-LX7
Panasonic DMC-TZ60
Panasonic DMC-TZ71
Pentax *ist D
Pentax *ist DL2
Pentax *ist DS
Pentax K100D Super
Pentax K10D
Pentax K20D
Pentax K-50
Pentax K-m
Pentax K-r
Pentax K-S1
Pentax Optio S4
Polaroid x530
Ricoh GR DIGITAL 2
Samsung EX2F
Samsung NX100
Samsung NX300
Samsung NX300M
Samsung NX500
Samsung WB2000
Sigma DP2 Quattro
Sigma DP1s
Sigma DP2 Merrill
Sigma SD10
Sigma SD14
Sigma SD9
Sony DSC-R1
Sony DSC-RX100
Sony DSC-RX100M2
Sony DSC-RX100M3
Sony DSC-RX100M4
Sony DSC-RX10
Sony DSC-RX10M2
Sony DSLR-A100
Sony DSLR-A200
Sony DSLR-A300
Sony DSLR-A330
Sony DSLR-A350
Sony DSLR-A550
Sony DSLR-A580
Sony DSLR-A700
Sony DSLR-A850
Sony DSLR-A900
Sony NEX-3
Sony NEX-5R
Sony NEX-7
Sony SLT-A35
Sony SLT-A58
Sony SLT-A77
Sony SLT-A99

We are really working hard to make sure we are a good resource of freely available raw samples for all Free Software imaging projects to use. Thank you so much for helping out if you can!


This is a companion discussion topic for the original entry at https://pixls.us/blog/2017/09/keep-the-raws-coming/

Nikon D4 import from lighttable
LEICA M Monochrom (Typ 246): RT cannot display DNG
#2

Note that some cameras have multiple names. Take the first item on the first list for example. It is also called Canon EOS 1000D and Canon EOS Rebel XS. Reference: https://en.wikipedia.org/wiki/Canon_EOS_1000D.


(Roman Lebedev) #3

Yes, a few entries in these two lists are actually about missing samples for aliases, not the main camera.
I think i need to look into tooling / better automation for creation of those two lists, and then maybe start a [META] thread with more-or-less valid / up-to-date lists, since that seems to be more user-friendly.


#4

Same for Canon EOS Rebel T4i and EOS 650D…


(Ingo Weyrich) #5

I will provide a Nikon D1x file licensed CC0 when I’m back at home in about a week.


(Bo Benson) #6

I uploaded a RW2 file from my Panasonic GM5, I just wonder why you need all aspect ratios as all but 4:3 is a crop?


(Roman Lebedev) #7

Thank you!

That is the point exactly. To have samples for all the possible raw file outputs.
Now, please upload the full set for that camera :slight_smile:
(i.e. upload new 4 samples, as-if that sample is not there)


(Ingo Weyrich) #8

As promised, I just uploaded a CC0 licensed Nikon D1x nef


(Roman Lebedev) #9

Please do note that camera can produce both compressed and uncompressed raw, and you have uploaded only one sample. Please upload the whole set :slight_smile:


(Ingo Weyrich) #10

I can not. I don’t own the camera any more.


(Roman Lebedev) #11

Ah, ok, thank you for that sample then :slight_smile:


(Ingo Weyrich) #12

Hah, I found an uncompressed file :slight_smile: Uploading now. CC0


(Roman Lebedev) #13

Nice :slight_smile: Thank you.


(Andreas Schneider) #14

I’ve uploaded an exmpale for the Nikon 1 J3. It only has a RAW option but no selection for other compression modes.


(Bruce Gordon) #15

In response to the problem of attacks or other hazards to the raw sample archive, has anyone considered using IPFS as a means of mitigating some of the risk? Just curious. Not an expert on it but intrigued by the concept.


(David Wilson) #16

I uploaded a Nikon D3S .nef file. If there are are other formats I can not help, I borrowed the camera and only took a few pictures with the same format. The camera has since been sold.


(Mica) #17

I don’t think we have considered IPFS, but I have a versioned backup of this data, so if something happened s to the main site, I can roll it back to a safe point.


(Roman Lebedev) #18

I have a versioned backup of this data, so if something happened s to the main site, I can roll it back to a safe point.

Thank you! :slight_smile: I also have fresh-ish checkouts, but that is not a backup.

Might also be good if @andabata would release the DB dumps.
I think they don’t contain any sensitive info. They would simplify recovery.


#19

Hi,

what type of images are actually needed for Nikon cameras? I uploaded the missing raw (.NEF) files for the Nikon D700, but someone remarked I should upload the full set. Does this mean the jpeg’s, too?

  • :white_check_mark: 12bit-compressed
  • :white_check_mark: 12bit-uncompressed
  • :white_check_mark: 12bit-lossless-compressed
  • :white_check_mark:14bit-compressed
  • :white_check_mark: 14bit-uncompressed
  • 14bit-lossless-compressed: already done by someone else in 2017.
  • small NEF: I don’t what a small NEF is and how to set it up in my camera.
  • medium NEF: I don’t what a medium NEF is and how to set it up in my camera.

(Roman Lebedev) #20

Me :wink:

If the camera can produce more than one sample, then we really really want all the samples to contain the same content. So

is ‘missing’ from your set.

(Also, would be great if all the samples are done with tripod, but well, better than nothing :slight_smile: )