I have simple task which I almost figured out but cannot get to end.
I have simple task - replace sky in random shot of train emerging from bridge. I was first trying to do it with chroma key but there was a problem as sky was both blue and white so it was mess (which was even slightly moving).
So I decided to go oldschool and using ColorCorrection and Invert nodes I created an matte (see fig 1 in my screenshot as seen from Viewer1). Then using Shuffle node I made alpha channel out of red and used original footage to get colors.
When I use this as first output and use my background plate as second output and use S-Over in Viewer3 it produces exactly what I am aiming for and it looks beautiful (see fig 2).
However when trying to replicate it with Merge (over) node I get really washed out sky instead of intended mountains (see fig 3 and Viewer3). It seems that I have set something bad but I am not sure what. I suspect that premultiplied/unpremultiplied business, but I have no idea what is it.
You can check if it’s a premult issue by adding a grade node to the matte and increasing the offset parameter. If you see any kind of change in the matte where there should be transparency, then you have a premult issue and you need to unpremult the matte. However I don’t think premult has got to do anything with the mountains. Either the shot doesn’t have enough data to help the keyer distinguish between the sky and the mountains or the matte needs more refinement. Hate to say this but if there isn’t enough data or if the shot is just exposed a certain way or if environmental conditions weren’t good when it was shot, the best thing to do would be to roto it.
I played with Natron little bit more and it seems that the following workflow is the general solution fot these types of tasks:
Firstly you need to have good contrast between foreground and background - background needs to be a lot brighter or darker than foreground. White wall, white paper, sky or dark curtain seems to do the trick for me. See Viewer1 in my new example.
Then use ColorCorrection or Grade to create black and white mask. You can use other nodes such as ErodeBlur or Invert to improve this mask. See Viewer2for how should mask look like - black is foreground.
Then use KeyerLuminance to convert it to pull alpha channel matte. You can use EdgeBlur to make it’s edges more blurry.
Then use MergeNode to bring it all together - connect it that A input if foreground, B input is background and Mask input is you newly created mask. Then use another MergeNode to remove alpha channel from it (it just need to be connected, nothing is needed to configure) and finally use Write to export finished product.
Ok so, sorry for replying after such a long time, I completely forgot I was involved in this thread. I see that you have figured out something and it is partially in the right direction. I would like to add a few points or suggestions:
This is what I meant by the shot not having “enough data”. Usually, unless you are saving a Raw file or a Log encoded file from a really good (think cine) camera, you’re not going to get a lot of data in your video file. Display-encoded files from DSLRs/MILCs/mobile phones (MPEG-4, AVC, etc) will usually compress the life out of the image. Combine that by shooting a low-contrast scene, or without good lighting means the camera will throw away even more data, and edges/seams with similar colors where you’d want the edge/seam to be prominent will get smudgy, making it difficult for a keyer to pick out the edge. This is why it’s necessary to create the perfect conditions for the camera and encoding that you’re going to use (hence why we light scenes artificially), or record at the highest quality available if you can’t do that. However, sometimes you can’t do both, and in that case you have to bite the bullet and go the roto route if you need a matte.
This step is unneeded, since it’s the job of the Keyer node in your next step to create the matte. You can however use the CC/Grade node to artificially increase the contrast in your shot, however be aware that doing so will also “enhance” any unwanted gremlins such as compression artifacts, banding, ghosting, flares, etc. making it harder for the keyer. The keyer has tolerance and threshold parameters to do these same adjustments, but using math that is specifically developed for the keyer.
If you can get a matte this way from the CC/Grade node, then you dont even require the keyer. All you have to do is swap the Alpha channel of your shot with the matte that you created using the CC/Grade node, using a shuffle node.
White should be the foreground, since the way an Alpha channel works is: anything that is white is visible, black is invisible and anything in the middle (gray values) is semi-transparent/translucent.
This should be the only step you need to do to get a matte, if your shot meets the ideal conditions. Sometimes, you’ll need a Roto node to make a garbage matte so that you’re not affecting regions that are you dont want to. Maybe an Erode or ErodeSmooth node after it to remove any residual pixels in the matte edges. The Keyer node has all the parameters that you need to get a key.
Here you can see how a mate can be made using only the keyer node (not counting the Roto and Erode since they are only there to help get a clean matte):
