Chris Healer Gets Into the Nitty-Gritty of Keying Over Black
Keying material over black screen can be a little tricky. With green screen, you’re relying on distance in chrominance between the screen and the subject. In black, you’re keying off luminance, kind of.
1. Before Image – Actor over black screen
2. Final Composite… With lots of fluffy hair.
The interior of the image will always have dark bits, which will fall out of your luma key, which can be fixed with rotos, but then we come to the real issue, which is how to get fluffy hairs to look right. We want to get a matte for this thing, and we don’t want to roto every frame of the shot.
3. Close-up of the frizzy hair over black. Notice how much residual noise is in the “black” area of the frame. Noise is always present, even on the darkest black screen.
4. The composite – The background has grain added to match the foreground.
It’s really important when shooing over a black screen that you try to edge light your subjects as much as possible. This reduces roto (especially precise roto) tremendously and will create a cleaner composite. Not all scenes allow it, but the ones that do will benefit from it.
We’ll need a luminance source, which in this case became the red channel because the skin and costume are redder than they are green or blue. The luminance of the image could have been used, but in this case it contains less contrast, so we chose red. The 4:2:2 color sampling creates a stair stepping effect, which isn’t really a problem after a little blur is added.
The matte at this point is expanded slightly to get as much matte as we can, so that as little as possible needs roto. The goal here is to be able to roto quickly without having to worry about fine detail, so the actual shapes end up being 10 or 20-pointed blobs that hover near the edge of the object.
After completing the rotos, we have a finished matte.
5. The red channel – Notice the 4:2:2 sampling artifacts from the Sony F900.
6. The red channel, Expanded to the maximum level we could get while still keeping the softness in the frizzy hairs.
7. A very loose roto, with almost no precise edge matching. Each of the 7 shapes probably only have 10 frames of roto.
8. The expanded red channel with the rotos added.
To comp this with the background we could do a regular premultiply and an ‘over’, but there is a better way. We really want to avoid premultiplication in order to keep the fine hairs and softness of the key. If you remember, an ‘over’ is:
Argb + (Brgb* (1 – Aa)) = Crgba
This means that the second image (the background) is multiplied by the inverted alpha of the first image (the foreground) and then added back to A. But the assumption here is that A is already premultiplied. So, for instance, in the transparent areas of the foreground image, alpha should be zero and rgb should be zero, which means that the background is totally unaffected. But what about the semi-transparent areas around the hair and small notches around the body? With a 50% alpha the rgb should be premultiplied to 50% its final brightness. Simple enough. But if we have a 50% alpha and the rgb is brighter than 50% (i.e. the image is un-premultiplied or it’s cross-faded with an un-premultiplied version), then the ‘over’ starts to become an ‘add’.
Why do we care about all this?
If we deconstruct the ‘over’ into an ‘outside’ (multiply) and an ‘add’ (inside a multilayer) then we have a chance to affect the images and alphas before they’re added. If you don’t want to premultiply the foreground (and thus keep all the hairs) then you’re going to have to remove the silhouette from the background before adding (image 10). Before doing this removal (an ‘outside’) we can shrink the matte a little bit, which reduces the alpha on the hairs (in some cases to nothing). This has the awesome effect of allowing the hairs to survive our faux ‘over’ operation. The important thing here is that we are effectively creating a situation where alpha doesn’t exist (or is a very low value) and the rgb is a value greater than zero (where the hairs are) so our ‘over’ in those areas starts to become an ‘add.’
It’s worth mentioning at this point that one could just use a ‘switch matte,’ with the shrunken matte, without premultiplication to achieve the same effect in an ‘over’ node, but it’s a little more graphical doing it the way described above, and easier to manipulate the pieces.
Because you’re not premultiplying, you have to expand your foreground a little to crush out the noise. Otherwise you’re doubling your grain and it gets a little messy.
9. The source footage expanded so that the noise crushes below zero black.
10. The background, “outside” the alpha.
Now we have a composite. Provided the rotos and luma-keys are good, it should hold up to motion without much (if any) sizzling or jittering. This is because we’re really not keying or premultiplying anything, just using ‘expands,’ ‘multiplys,’ ‘overs’ and ‘adds’ to get there.
11. The composite without light wrap.
And what composite would be complete without a light wrap? Blurring the background with the silhouette cut out gives us our wrap source, which will respond as the footage changes (the waves cover the sun for a moment). Doing an inside on our alpha and ‘adding’ that to our composite creates the final composite.
12. Light wrap.
13. Light wrap inside the alpha.
14. Composite without light wrap.
15. Composite with light wrap.
16. Final composite.
17. Final Composite
The shake tree is pretty simple, and in practice an un-premultiplied expanded RGB with an alpha channel will save you lots of hairs, both in your comp and on your head!
18. The Shake tree is pretty simple, really.