Chroma key

"Green screen" redirects here. For other uses, see Green screen (disambiguation).

For the electronic music project, see Chroma Key. For musical tonality depending on key, see Key (music).

Today's practicality of green-screen compositing is demonstrated by Iman Crosson in a self-produced YouTube video.
Top panel: A frame of Crosson in full-motion video as shot in his own living room.
Bottom panel: Frame in the final version, in which Crosson, impersonating Barack Obama, "appears" in the White House's East Room.

Chroma key compositing, or chroma keying, is a special effects / post-production technique for compositing (layering) two images or video streams together based on color hues (chroma range). The technique has been used heavily in many fields to remove a background from the subject of a photo or video – particularly the newscasting, motion picture and videogame industries. A color range in the top layer is made transparent, revealing another image behind. The chroma keying technique is commonly used in video production and post-production. This technique is also referred to as color keying, colour-separation overlay (CSO; primarily by the BBC), or by various terms for specific color-related variants such as green screen, and blue screen – chroma keying can be done with backgrounds of any color that are uniform and distinct, but green and blue backgrounds are more commonly used because they differ most distinctly in hue from most human skin colors. No part of the subject being filmed or photographed may duplicate a color used in the background.

It is commonly used for weather forecast broadcasts, wherein a news presenter is usually seen standing in front of a large CGI map during live television newscasts, though in actuality it is a large blue or green background. When using a blue screen, different weather maps are added on the parts of the image where the color is blue. If the news presenter wears blue clothes, his or her clothes will also be replaced with the background video. A complementary system is used for green screens. Chroma keying is also used in the entertainment industry for special effects in movies and videogames. The advanced state of the technology and much commercially available computer software, such as Autodesk Smoke, Final Cut Pro, Pinnacle Studio, Adobe After Effects, and dozens of other computer programs, makes it possible and relatively easy for the average home computer user to create videos using the "chromakey" function with easily affordable green screen or blue screen kits.

History

In filmmaking, a complex and time-consuming process known as "travelling matte" was used prior to the introduction of digital compositing. The blue screen method was developed in the 1930s at RKO Radio Pictures. At RKO, Linwood Dunn used an early version of the travelling matte to create "wipes" – where there were transitions like a windshield wiper in films such as Flying Down to Rio (1933). Credited to Larry Butler, a scene featuring a genie escaping from a bottle was the first use of a proper bluescreen process to create a traveling matte for The Thief of Bagdad (1940), which won the Academy Award for Visual Effects that year. In 1950, Warner Brothers employee and ex-Kodak researcher Arthur Widmer began working on an ultraviolet travelling matte process. He also began developing bluescreen techniques: one of the first films to use them was the 1958 adaptation of the Ernest Hemingway novella, The Old Man and the Sea, starring Spencer Tracy.

One drawback to the traditional traveling matte is that the cameras shooting the images to be composited cannot be easily synchronized. For decades, such matte shots had to be done "locked-down", so that neither the matted subject nor the background could shift their camera perspective at all. Later, computer-timed, motion-control cameras alleviated this problem, as both the foreground and background could be filmed with the same camera moves. Petro Vlahos was awarded an Academy Award for his refinement of these techniques in 1964. His technique exploits the fact that most objects in real-world scenes have a color whose blue-color component is similar in intensity to their green-color component. Zbigniew Rybczyński also contributed to bluescreen technology. An optical printer with two projectors, a film camera and a 'beam splitter', was used to combine the actor in front of a blue screen together with the background footage, one frame at a time. In the early 1970s, American and British television networks began using green backdrops instead of blue for their newscasts. During the 1980s, minicomputers were used to control the optical printer. For the film The Empire Strikes Back, Richard Edlund created a 'quad optical printer' that accelerated the process considerably and saved money. He received a special Academy Award for his innovation.

For Star Trek: The Next Generation, an ultraviolet light matting process was proposed by Don Lee of CIS and developed by Gary Hutzel and the staff of Image G. This involved a fluorescent orange backdrop which made it easier to generate a holdout matte, thus allowing the effects team to produce effects in a quarter of the time needed for other methods.

Meteorologists on television often use a field monitor, to the side of the screen, to see where they are putting their hands against the background images. A newer technique is to project a faint image onto the screen.

Some films make heavy use of chroma key to add backgrounds that are constructed entirely using computer-generated imagery (CGI). Performances from different takes can even be composited together, which allows actors to be filmed separately and then placed together in the same scene. Chroma key allows performers to appear to be in any location without even leaving the studio. The comedy show Tosh.0 in particular is known for filming entirely with greenscreen and then cutting the image being produced by the computer to show Daniel Tosh standing directly on the purely green set. This is always done at the very end of the show.

Computer development also made it easier to incorporate motion into composited shots, even when using handheld cameras. Reference-points can be placed onto the colored background (usually as a painted grid, X's marked with tape, or equally spaced tennis balls attached to the wall). In post-production, a computer can use the references to compute the camera's position and thus render an image that matches the perspective and movement of the foreground perfectly. Modern advances in software and computational power have even eliminated the need to accurately place the markers - the software figures out their position in space (a disadvantage of this is that it requires a large camera movement, possibly encouraging modern film techniques where the camera is always in motion).

Process

Film set for The Spiderwick Chronicles, where a special effects scene using bluescreen chroma key is in preparation.

The principal subject is filmed or photographed against a background consisting of a single color or a relatively narrow range of colors, usually blue or green because these colors are considered to be the furthest away from skin tone. The portions of the video which match the preselected color are replaced by the alternate background video. This process is commonly known as "keying", "keying out" or simply a "key".

Processing a green backdrop

Virtual television studio with green-screen technique.

Green is currently used as a backdrop more than any other color because image sensors in digital video cameras are most sensitive to green, due to the bayer pattern allocating more pixels to the green channel, mimicking the human eye's increased sensitivity to green light. Therefore, the green camera channel contains the least "noise" and can produce the cleanest key/matte/mask. Additionally, less light is needed to illuminate green, again because of the higher sensitivity to green in image sensors. Bright green has also become favored since a blue background may match a subject's eye color or common items of clothing such as jeans.

Processing a blue backdrop

Before digital chroma keying, bluescreening was accomplished using film. The camera color negative was printed onto high-contrast black and white film, using either a filter or the color sensitivity of the black and white film to limit it to the blue channel. Assuming this film was a negative it produced clear where the bluescreen was, black elsewhere, except it also produced clear for any white objects (since they also contained blue). Removing these spots could be done by a suitable double-exposure with the color positive, and many other techniques. The end result was a clear background with an opaque shape of the subject in the middle. This is called a 'female matte', similar to an 'alpha matte' in digital keying. Copying this film onto another high-contrast negative produced the opposite 'male matte'. The background negative was then packed with the female matte and exposed onto a final strip of film, then the camera negative was packed with the male matte was double-printed onto this same film. These two images combined together creates the final effect.

Blue was preferred as a backdrop before digital keying became commonplace because of the existence of high contrast film that was sensitive only to the blue color.

Major factors

The most important factor for a key is the color separation of the foreground (the subject) and background (the screen) – a bluescreen will be used if the subject is predominately green (for example plants), despite the camera being more sensitive to green light.

In analog color TV, color is represented by the phase of the chroma subcarrier relative to a reference oscillator. Chroma key is achieved by comparing the phase of the video to the phase corresponding to the preselected color. In-phase portions of the video are replaced by the alternate background video.

In digital color TV, color is represented by three numbers (red, green, blue intensity levels). Chroma key is achieved by a simple numerical comparison between the video and the preselected color. If the color at a particular point on the screen matches (either exactly, or in a range), then the video at that point is replaced by the alternate background

Clothing

Girl wearing blue clothing in front of green screen.

A chroma key subject must avoid wearing clothes which are similar in color to the chroma key color(s) (unless intentional e.g. wearing a green top to make it appear that the subject has no body), because the clothing may be replaced with the background video. An example of intentional use of this is when an actor wears a blue covering over a part of his body to make it invisible in the final shot. This technique can be used to achieve an effect similar to that used in the Harry Potter films to create the effect of an invisibility cloak. The actor can also be filmed against a chroma-key background and inserted into the background shot with a distortion effect, in order to create a cloak that is marginally detectable.

Difficulties emerge with bluescreen when a costume in an effects shot must be blue, such as Superman's traditional blue outfit. In the 2002 film Spider-Man, in scenes where both Spider-Man and the Green Goblin are in the air, Spider-Man had to be shot in front of the greenscreen and the Green Goblin had to be shot in front of a bluescreen. The color difference is because Spider-Man wears a costume which is red and blue in color and the Green Goblin wears a costume which is entirely green in color. If both were shot in front of the same screen, parts of one character would be erased from the shot.

Background

Demonstration of the creation of special effects techniques utilizing chroma key.

Blue is generally used for both weather maps and special effects because it is complementary to human skin tone. The use of blue is also tied to the fact that the blue emulsion layer of film has the finest crystals and thus good detail and minimal grain (in comparison to the red and green layers of the emulsion.) In the digital world, however, green has become the favored color because digital cameras retain more detail in the green channel, and it requires less light than blue. Green not only has a higher luminance value than blue, but also in early digital formats, the green channel was sampled twice as often as the blue, making it easier to work with. The choice of color is up to the effects artists and the needs of the specific shot. In the past decade, the use of green has become dominant in film special effects. Also, the green background is favored over blue for outdoor filming where the blue sky might appear in the frame and could accidentally be replaced in the process. Although green and blue are the most common, any color can be used. Red is usually avoided due to its prevalence in normal human skin pigments, but can be often used for objects and scenes which do not involve people. For example, in John Pizzarelli's song Birthday Emotions from the Sesame Street television series, painting backgrounds made by Gerri Brioso are used as a live-action film sequence while the kids such as the Italian-American siblings are jumping through the air and celebrating one of their birthday parties.

Occasionally, a magenta background is used, as in some software applications where the magenta or fuchsia is sometimes referred to as "magic pink".

With other imaging and hardware, many companies avoid the confusion often experienced by weather presenters, who must otherwise watch themselves on a monitor to see the image shown behind them, by lightly projecting a copy of the background image onto the blue/green screen. This allows the presenter to accurately point and look at the map without referring to monitors.

A newer technique is to use a retroreflective curtain in the background, along with a ring of bright LEDs around the camera lens. This requires no light to shine on the background other than the LEDs, which use an extremely small amount of power and space unlike big stage lights, and require no rigging. This advance was made possible by the invention of practical blue LEDs in the 1990s, which also allow for emerald green LEDs.

There is also a form of color keying that uses light spectrum invisible to human eye. Called Thermo-Key, it uses infrared as the key color, which would not be replaced by background image during postprocessing.

Jefferson Airplane used chroma-key background in their performance of "White Rabbit" on "The Smothers Brothers Comedy Hour" to create psychedelic colors.

Even lighting

A live broadcast of Myx TV using green-screen chroma key.

The biggest challenge when setting up a bluescreen or greenscreen is even lighting and the avoidance of shadow, because it is best to have as narrow a color range as possible being replaced. A shadow would present itself as a darker color to the camera and might not register for replacement. This can sometimes be seen in low-budget or live broadcasts where the errors cannot be manually repaired. The material being used affects the quality and ease of having it evenly lit. Materials which are shiny will be far less successful than those that are not. A shiny surface will have areas that reflect the lights making them appear pale, while other areas may be darkened. A matte surface will diffuse the reflected light and have a more even color range. In order to get the cleanest key from shooting greenscreen it is necessary to create a value difference between the subject and the greenscreen. In order to differentiate the subject from the screen, a two-stop difference can be used, either by making the greenscreen two stops higher than the subject, or vice versa.

Sometimes a shadow can be used to create a special effect. Areas of the bluescreen or greenscreen with a shadow on them can be replaced with a darker version of the desired background video image, making it look like the person casting. Any spill of the chroma key color will make the result look unnatural. Even a difference in the focal length of the lenses used can affect the success of chroma key.

Exposure

Another challenge for bluescreen or greenscreen is proper camera exposure. Underexposing or overexposing a colored backdrop can lead to poor saturation levels. In the case of video cameras, underexposed images can contain high amounts of noise, as well. The background must be bright enough to allow the camera to create a bright and saturated image.

Programming

There are several different quality- and speed-optimized techniques for implementing color keying in software.

In most versions, a function f(r, g, b) → α is applied to every pixel in the image. α (alpha) has a meaning similar to that in alpha compositing techniques. α = 0 means the pixel is fully in the green screen, α = 1 means the pixel is fully in the foreground object, and intermediate values smooth away any halo at the border of object and screen.

A straightforward approach is to describe two closed nested surfaces in color space and use the color distance of each pixel from the key color to separate background, transition and foreground elements:

• Convert the pixel values into the YCbCr color space, effectively separating the luminance from the chrominance color information — hence, "chroma" key. (Conveniently, this is the native color space popular video formats such as MPEG.)
• Determine the Euclidean distance between each pixel's color ("Cb" and "Cr" coordinates) and a defined ("Cb", "Cr") constant representing the screen color
• If the distance is within an inner threshold (i.e. very close to the screen color), it's considered to be entirely within the background -- α = 0
• If the distance is above an outer threshold, it's considered to be entirely within the foreground -- α = 1
• In between the inner and outer thresholds, do a linear interpolation from 0 to 1: ("dist"-"inner")/("outer"-"inner"). That function matches the capped values of 0 at the inner threshold and 1 at the outer threshold.

Professional keyers use far more sophisticated functions. It is also very common for f() to depend on more than just the current pixel's color, it may also use the (x, y) position, the values of nearby pixels, the value from reference images, and values from user-drawn masks.

A different class of algorithm tries to figure out a 2D path that separates the foreground from the background. This path can be the output, or the image can be drawn by filling the path with α = 1 as a final step. An example of such an algorithm is the use of active contour. Most research in recent years has been into these algorithms.

See also

• Compositing
• Drew Carey's Green Screen Show
• Federal Standard 1037C
• Filmmaking
• Film production
• Front projection effect
• Live action
• Live action/Animation

• Matte (filmmaking)
• Motion control photography
• Muslin
• Optical printer
• Primatte chromakey technology
• Rear projection effect
• Reverse bluescreen

• Schüfftan process
• Signal processing
• Sodium vapor process
• Special effects
• Video switcher
• Virtual set

External links

• Low cost green screen software
• How Blue Screens Work
• Stargate Studios Virtual Backlot Reel 2009 -- A demonstration of green screen scenes
• ChromaKey SourceCode
• Safire 3 real-time chroma keyer