How to create synthetic RAW image?

I am looking for a way to create synthetic raw image for testing purposes.

How can I do that?

(Or should I ask in a different forum?)

That depends in what you want to test with those. Do you just need synthetic Bayer data or do you want to “fake” a whole raw file in some camera’s file format? I haven’t done either but the latter sounds really hard if not impossible, depending on the amount of detail you want to get.

Hmm. That raises an interesting question. If someone steals a picture and sells it as its own, can the original author go to court with the raw version as the sole proof of ownership, or it is possible for the thief to come up with a retro-computed raw file which, with at least one demosaicing algorithm and adequate combination of parameters, yields the published picture?

Do you mean generating synthetic test raw files such as the raw file used in this article?

Bayer moire:
https://www.libraw.org/articles/bayer-moire.html

I need this in order to test particular post-production procedures (including raw conversion an perhaps even demosaic) but because I want this to be a very controlled test I cannot achieve the accuracy I need using real photographic data. For example I want to be able to create particular image data which I don’t want to be influenced by things like lens imperfections and other technical factors BUT still I want to be able to simulate the whole process itself as if it is on a real raw image.

Yes, perhaps something like this but for more advanced tests. How are these generated?

FWIW I have already sent an email to libraw’s creator but I am not getting any reply for the moment.

My apologies, I don’t know how such synthetic images are created. But synthetic images would be very useful for testing and comparing raw processing algorithms.

The “moire” image definitely is useful for comparing demosaiking algorithms, but as far as I can tell it’s not released under a free/libre compatible copyright.

It would be nice to be able to generate synthetic scene data as well as synthetic test chart type data.

Hopefully someone who knows will see this thread.

Maybe try one of the dpreview.com forums?, e.g. “photo. science and tech.”

Thanks for the suggestion. I have done that too.

BTW not sure who should be notified about this but all email notifications sent by this forum end up in the Spam folder in Gmail. When looking at the headers I see that SPF: softfail and there are no DKIM and DMARC records which is obviously the reason. I hope someone who takes care of this will look into it.

This is too vague. If you simply want the mosaiced image, then it’s trivial (e.g creation of DNG is pretty simple). If you want to generate the whole as-if camera raw, then it could be as complicated as acquiring a camera and messing with it’s electronics and firmware in order to trick it into compressing your data into the camera-specific compression/etc.

I am willing to be able to control the hardware WB (uniwb) add noise or not, generate particular pixel patterns, gradients and other things, and obviously that should be mosaiced and linear so that it is like a real raw. I am not looking to simulate the electronics of a particular camera body but rather (as explained) to create data free from these particular influences (which we usually consider imperfections).

Do you know how to do this?

Well, if you can generate the data, containering it into DNG is trivial

Thanks but I haven’t touched C code for 15 years, so I am not quite sure what to do with that :slight_smile: If you can explain how to turn this into executable code hopefully I may give it a try. (I am on Linux)

You could create a DNG file using ExifTool and ImageMagick/GraphicsMagick. If you want it mosaiced, I think G’MIC has a filter.
http://rawpedia.rawtherapee.com/Film_Simulation#Advanced_-_Identity_DNG

Thanks. That’s interesting and sounds easy. But after running the 3 command lines and opening the resulting DNG in RawTherapee it looks strange (as if wrong):

Do you have any idea why? (I haven’t applied any G’MIC filter)

G’MIC does have a filter, -rgb2bayer. Go here:

http://gmic.eu/reference.shtml

Search for ‘rgb2bayer’. Doesn’t describe specifically what the transform is, but would be a good start.

George has an equivalent thread going at dpreview. Based on the discourse there, the workflow that makes sense to me would be:

  1. Make the desired synthetic image in whatever drawing program, save as TIFF
  2. Use G’MIC to do the ‘reverse-engineering’ to a raw:
  • de-scale
  • white/black point
  • rgb2bayer
  • save as TIFF
  1. Maybe use exiftool to write the TIFF into the image array of a .DNG (I’m less familiar with this use of exiftool)

Yes, quite a lot of words on dpreview but still no clear methodology.

What is de-scale and white/black point? I have never used G’MIC but looking at the online filter gallery of G’MIC I don’t see those.

Also how do you write the TIFF to DNG? See above what happens when I used exiftool for this.

Works fine here:

de-scale: The image you create in whatever program will contain pixel values scaled to the full extent of the data type, e.g., 0-255 for 8-bit values, accommodating the behavior of human vision. The raw image from a camera looks dark to the human eye because the sensor records the light values linearly, e.g., white in a raw I just opened without applying processing is 64,64,64 in a 0-255 extent. So, you’ll need to essentially apply ‘reverse-gamma’ to the synthetic image, where 2.2 is nominally the gamma applied when raw/linear is converted to sRGB, so do the inverse. G’MIC may be able to do this with -adjust_colors, which has a gamma input.

black/white point: you may not have to do this for a synthetic image, if the reverse gamma does what I think it should.

Generally, G’MIC is the tool I believe to be most suited to your needs, but like any powerful tool, you’ll have to learn it. The documentation is a bit obtuse, but it’s cheap to craft a command, run it, and see what it does. Also, David Tschumperlé, the primary programmer behind G’MIC, posts here and would probably be very happy to answer specific questions about it.