CEIG’09, San Sebastián, Sept.

9-11 (2009)
C. Andújar and J. LLuch (Editors)

Light source detection in photographs

Jorge Lopez-Moreno1 , Sunil Hadap2 , Erik Reinhard3 and Diego Gutierrez 1

1 Universidad de Zaragoza 2 Adobe Systems Inc 3 University of Bristol

Abstract

Common tasks related to image processing or augmented reality include rendering new objects into existing ima-
ges, or matching objects with unknown illumination. To facilitate such algorithms, it is often necessary to infer
from which directions a scene was illuminated, even if only a photograph is available. For this purpose, we present
a novel light source detection algorithm that, contrary to the current state-of-the-art, is able to detect multiple light
sources with sufficient accuracy. 3D measures are not required, only the input image and a very small amount of
unskilled user interaction.
Categories and Subject Descriptors (according to ACM CCS): Computing Methodologies [I.3.7]: Computer
Graphics—3D Graphics; Computing Methodologies [I.4.10]: Image Processing and Computer Vision—Image
Representation

1. Introduction ject in the image as a de-facto light probe. We have found

that assuming a globally convex shape for such light probe
This paper deals with the problem of obtaining the positions
is sufficient to reconstruct light directions. The user only
and relative intensities of light sources in a scene, given only
needs to provide its silhouette in the image, a task sim-
a photograph as input. This is generally a difficult and under-
ilar or simpler than other existing image editing applica-
constrained problem, even if only a single light source illu-
tions [OCDD01, LHE∗ 07]. We then analyze the information
minates the depicted environment.
in the contour and the gradients contained in the shape to
Traditionally light probes are used to acquire the light- infer the light directions and relative intensities.
ing data. A light probe is an object of known 3D shape Real environments are likely to contain multiple light
and BRDF properties (Bidirectional Reflectance Distribu- sources. We found that in practice finding up to four sources,
tion Function; description of the reflectance properties of the combined to provide similar illumination as in the image,
material) which is positioned in the scene when the image is suffices for most situations. This keeps the dimensionality
captured. Unfortunately, in several cases this technique is not of the solution manageable, in a way similar to profession-
applicable: e.g. paintings, photographs taken under uncon- ally lit environments, which are usually lit by a three-light
trolled conditions, etc. Instead, it would be possible to use setup. Additionally, although we do assume in principle that
any object in the image if geometry information were avail- the chosen light probe is Lambertian, we will show that this
able, allowing light source positions or directions to be esti- is not a strong requirement.
mated [GHH01]. Conversely, if the light source is known,
the 3D geometry can be approximately recovered, an ill- We believe that by analyzing lighting consistency between
posed problem known as shape-from-shading [ZTCS99]. images our algorithm can help improve several types of ap-
plications, such as Photo Clip Art [LHE∗ 07], Interactive
However, we are interested in the problem of light source Digital Photomontage [ADA∗ 04] or Photo Tourism [SSS06].
recovery without the benefit of any geometric prior. To this
end, we first carry out a psychophysical experiment to quan-
2. Previous Work
tify the accuracy with which humans can generally detect
light sources. The results of this experiment are then used The computation of light source directions from images is
to validate the results of our light detection algorithm, both an ill-posed problem, with many possible solutions leading
numerical and perceptually. We then use any existing ob- to the same observed image. As a result, assumptions about

c The Eurographics Association 2009.
 Jorge Lopez-Moreno & Sunil Hadap & Erik Reinhard & Diego Gutierrez / Light source detection in photographs

the environment are made, known geometry must be present

in the scene, or extra information must be captured to change
the problem into a solvable one.
For detecting single light sources, by itself a strong as-
sumption on the environment, a local analysis of the surface
and image derivatives may be used to estimate the direc-
tion of the light source [Pen82, BH85, LR89]. Alternatively,
occluding contours within a single object [Hor86, NE01] or
texturing [KP03, VZ04] provide clues as to where the light
is coming from.
To detect multiple lights, the environment could be pho-
tographed along with the aforementioned lightprobe: a cali-
bration object of known size and shape. For instance, a Lam-
bertian sphere could be employed, and subsequently ana-
lysed for multiple light source directions [HA93, ZY01], or
alternatively multiple specular spheres can be triangulated
for the same purpose [PSG01,LF06]. Combinations of Lam-
bertian and specular spheres have also been used [ZK02]. Figura 1: a) Input Image b) Object c) Silhouette Normals d)
Finally, reflections of a human eye can be analysed to detect Coordinate System
light sources [NN04].
Another way to overcome the under-constrained nature of
the problem is to use a range camera to record geometry, a- 1. To estimate the number of lights N and their respective
llowing light sources to be inferred from the combination of azimuth coordinates φi , i = 1 · · · N; we analyze the inten-
photograph and range data [MG97]. Known geometry can be sity variation along the silhouette of the object. We as-
used to the same effect [WS02, SSI99]. In contrast, our ap- sume that the surface normals of the object at the sil-
proach is free of previous restrictions, e.g.: there is no need houette lie in the image plane [Hor86]. Using the silhou-
for a calibration object or known geometry. Furthermore, we ette normal assumption and the nominal diffuse lighting
do not require shadows being cast on nearby objects, nor is equation, we can accurately predict the azimuth coordi-
any camera information needed. nate φ of the individual lights. The number of lights and
their relative intensities are estimated in an iterative fash-
3. Light Detection ion.
2. We use the globally convex assumption to estimate the
Consider a typical input image as depicted in Figure 1a. The zenith angles θi , i = 1 · · · N and relative intensities Ii . For
problem at hand is to estimate the number of illumination each light detected in the first step, we sweep the im-
sources, their dominant directions and the relative intensi- age from the silhouette to the interior along the azimuth
ties. We propose to use any object in the image as a virtual direction, looking for maxima in the shading. The cor-
light-probe, as long as it covers a reasonable area in the i- responding shape normal at the maxima ~ni is indicative
mage. The user provides the outline defining the object, ty- of the direction of the light and thus the zenith angle
pically with the aid of a smart selection tool [WAC07]. We θi . To robustly handle local non-convexities and back-
do not assume any restrictions on the shape, the color or any lighting we detect and use shadows. Following Khan et
other peculiarities for the object. al. [KRFB06], we differentiate the relatively high fre-
quency variations of the luminance due to albedo (tex-
ture), from the low frequency variations of luminance due
3.1. Assumptions and Overview
to shading, using bilateral filtering.
To achieve a reasonable solution, we rely on the particular 3. By analyzing median intensity in the shadow areas, we
characteristics of human vision. In estimating illumination, estimate the ambient light intensity.
the human visual system tends to ignore the local shape
Each of these steps are explained in detail in the following
variations, and treats the object as a globally convex geo-
sections.
metry [LB01]. We also leverage the tendency of the human
visual system to perceive objects correctly as long as the i- However, we start by defining the coordinate system used.
llumination is locally consistent [OCS05]. Further, we ob- As depicted in Figure 1d, the image plane is assumed to be
serve that humans are surprisingly good at estimating back- aligned with the y−z plane, whereas the x axis points out of
lighting using cues from shadows [KvDP04]. Based on these the image plane. The origin lies at the center of the image.
assumptions, we devise a three-step algorithm as follows. We also set a polar coordinate system such that the equator

c The Eurographics Association 2009.
 Jorge Lopez-Moreno & Sunil Hadap & Erik Reinhard & Diego Gutierrez / Light source detection in photographs

is aligned with the image plane and the axis is aligned with Algorithm 1 Contour Voting - N lights
x axis. Thus the direction of a light is uniquely identified by Require: Lv ≡ {Liv } {discrete luminances}
the azimuth angle φ and the zenith angle θ . Require: n ≡ {ni } {silhouette normals}
Require: φ n ≡ {φin } {azimuth coordinates of the nor-
3.2. Estimating Azimuth Angles mals}
1: sort( Lv , n, φ n ) {sort by decreasing luminances}
We assume that the normals at the silhouette lie in the image 2: φ l ≡ {φ lj } | j ∈ [1 · · · N] {azimuth coordinates of the
plane. We further assume that there are N discrete lights, lights}
each being either a directional light or a far away point light 3: seed( φ l )
(we estimate N below). Thus each light is uniquely charac- 4: α ⊕ ≡ {α ⊕ j } | j ∈ [1 · · · N] {aggregate of weights per
terized by its unknown luminance L j and unknown unit di- light}
rection ω j , j = 1 · · · N. To analyze the intensity variation of 5: α ⊕ ← 0
the silhouette pixels, we assume a nominal Lambertian sur- 6: repeat
face. Consider all pixels {pi } that belong to the silhouette. 7: for all Liv ∈ Lv do
Let ni be the normal and Liv be the known luminance of the 8: ω j ← [0, sin(φ lj ), cos(φ lj )]T {current direction}
object at point pi :
9: Ω⊕ i ← ∑ j Ω(ni , ω j ) {total occlusion weight}
N 10: for all j ∈ [1 · · · N] do
Liv = ∑ Ωi j L j (1) 11: ω j ← [0, sin(φ lj ), cos(φ lj )]T {current direction}
j=1
(
0 if ni ·ω j < 0, 12: αi j ← Liv Ω(ni , ω j )/Ω⊕
i {weight of normal i}
Ωi j = Ω(ni , ω j ) =
Kid ni ·ω j if ni ·ω j ≥ 0 13: φ lj ← α ⊕ φ
j j
l + α φ n {update direction}
ij i

where Ω represents the occlusion function of pixel i 14: α⊕ ⊕

j ← α j + αi j
in the viewing direction and Kid is the unknown dif- 15: φ lj ← φ lj /α ⊕j
fuse reflectivity or albedo. We encode the normals, which 16: end for
are in the y−z plane, as polar coordinates φin → ni = 17: end for
[0, sin(φin ), cos(φin )]T , 0 ≤ φin ≤ 2π. 18: until convergence(φ l )
19: return φ l
To estimate the lights’ azimuth angles φ lj , we use a k-
means clustering algorithm. In traditional k-means cluster-
ing algorithms, each data point belongs to a certain cluster,
and affects the centroid of only that cluster. Unfortunately, a
will subsequently increase the number of lights N = 1..i until
silhouette pixel may be illuminated by more than one light.
either the error is below a given tolerance or the added light
Thus, we should not partition the pixels into exclusive clus-
source does not improve the result. In practice, we found
ters. Instead, we devise a partial voting scheme based on the
that the number of iterations is usually below N = 4. This is
occlusion function to form the ‘fuzzy’ clusters, and simulta-
due to the quantization associated with the image’s finite bit-
neously compute the corresponding centroids as the lighting
depth. As the number of opposed lights increases, the varia-
directions, as outlined in Algorithm 1.
tion in the shading over the surface decreases and becomes
We go through the list of pixels sorted by luminance (line rather constant.
7) to perform the normal voting. Notice that each silhou-
Although the proposed voting method has built-in resis-
ette normal φin votes all the N light clusters (lines 10 to
tance to local variations in albedo due to its search of global
16), according to their luminances Liv . However, each nor-
tendencies, ultimately the results are biased if the points in
mal only partially votes to each light cluster according to
the contour form large clusters with very different luminance
the occlusion function (line 12). For that, the individual oc-
values as the first image of Figure 2a demonstrates.
clusion function with respect to each light direction Ωi j is
normalized with the aggregate of occlusion functions Ω⊕ i = It is possible to reduce this bias, in a second pass, as fol-
∑ j Ω(ni , w j ). lows. Once we have a set of N centroids (light directions),
We repeat the voting process (line 7 to 17) until we con- we go through all the voting pixels assigned to each k-group,
verge on the light azimuth angles φ l (line 6 and 18). The corresponding to a light direction. We then check that the
choice of the initial guess (line 3) for the azimuth angles is dot product of the normal and the estimated light direction
important to ensure a speedy and effective convergence. We yields a luminance value equal to the original luminance of
assign the azimuth of the brightest pixel’s normal φ1n to the the pixel, fractioned by its occlusion function. If not, we
force the fractional albedo of the pixel to be coherent with
first light φ1l . For the successive lights, we set the azimuth
the fractional luminance of the brightest pixel in the group.
angles to φ1l + 2π( j − 1)/N.
Then we repeat the contour voting algorithm. This correc-
For the estimation of the number of lights N, our approach tion in albedo values usually produces small shifts (10 to 20

c The Eurographics Association 2009.
 Jorge Lopez-Moreno & Sunil Hadap & Erik Reinhard & Diego Gutierrez / Light source detection in photographs

a b a b

Figura 2: a1 ) Sphere with a change in the albedo a2 ) Initial

biased estimation due to higher albedo a3 ) Corrected light
direction estimate b1 ) An estimate incorrectly biased due to Figura 3: Estimating zenith angle a) Scanning in lights di-
the geometry of the silhouette b2 ) The correct result, after rection for highlight or shadow b) Ellipsoidal geometry
eliminating multiple normals.

increase as we march along the direction of the light, to reach

degrees) in the directions in the case of extreme albedo vari- the first local maximum. We denote this point as phij . At this
ations (Figure 2a). point the surface normal points in the direction of the light,
e.g.: θ j = θ n (phij ). We ignore all the pixels thereafter as the
As in other previous approaches based on contour analy-
geometry might be self-occluding or under the influence of
sis [YY91, VY94, NE01], the first step will fail if the light is
another light.
situated around the x axis, e.g.: θ ≈ π/2. In this case there
is no variation in luminances due to shading. This would re- Case 2: At the silhouette, if the directional derivative is
sult in erroneous estimation of azimuth angles. However, the negative, this is an indication of back lighting (θ < 0). The
final direction of the light would be estimated accurately in luminances will successively decrease as we march along
the second step when we analyze the shading in the interior. the light direction to reach a singularity. This point is the
first self-shadow point ploj , and is marked by either a change
Finally, we correct the potential bias along the direction
of sign in the gradient of the directional derivative ω j ·∇Lv
stemming from the geometry of the silhouette. As depicted
or a zero value of its luminance Lv . A change of sign will
in Figure 2b, a significant number of silhouette normals
be produced when the contribution to the luminance value at
point in y axis, biasing the resultant light towards that di-
that point by a second light is greater than the contribution
rection. We propose to correct this by eliminating multiple
of Lv . At this point, the surface normal is perpendicular to
normals. We choose a set of discrete normal directions φ̄in
the light direction, e.g.: θ j − θ n (plo
j ) = π/2, θ j < 0.
and distribute all the silhouette normals into bins. Then we
compute the average of luminances for each bin L̄i and use To estimate the normal at each point we cannot rely
this set of silhouette normals and luminances instead. on shape-from-shading due to the overlapping of multiple
lights. It is not possible to know a priori which combina-
tion of light sources is contributing to a certain point. Good
3.3. Estimating Zenith Angles and Intensities
solutions for estimate a valid normal at points phij or ploj in
To estimate zenith angles {θ j } accurately, we disambiguate arbitrary images do not exist [ZTCS99].
the luminance variations due to shading from the variations
Furthermore, this is complicated if two given points on
due to texture, which are relatively high in frequency. We use
the surface of the object are lit by a different and unknown
bilateral filtering to remove high frequencies, while keep-
number of light sources. Wang et al. [WS02] determined the
ing lower frequency content which is typically attributed to
number of lights, but they could perform this thanks to ac-
shading [KRFB06].
curate knowledge of 3D depth and normals. Instead, we re-
Then, for each light detected in the previous step, march- vert once more to our global convexity assumption and fit an
ing in the light’s direction ω j = ω(φ lj ) from the silhouette ellipse along the scanline: one of the axis is given by the in-
to the interior, we analyze the luminances. As the pixels are tersection of such scanline and the silhouette; the other axis
lit by multiple lights, this directional derivative of the lumi- will approximate the object convexity and is a user parame-
nance ω j ·∇Lv is the main indicator of the shading due to a ter. By default, both axes are equal (in fact defining a circum-
particular light j aligned to its direction. There are two cases ference). The surface normal is subsequently assumed to be
of luminance variations in the interior. the normal of the ellipse at the point under consideration.
Case 1: If the directional derivative ω j ·∇Lv is positive We could start marching along the light direction from a
at the silhouette, the light is towards the camera from the brightest silhouette point that corresponds to the light. How-
image (θ ≥ 0). In this case, the luminances will continue to ever, in order to minimize the influence of albedo variations,

c The Eurographics Association 2009.
 Jorge Lopez-Moreno & Sunil Hadap & Erik Reinhard & Diego Gutierrez / Light source detection in photographs

we scan the light direction from multiple silhouette points. errors usually below 20 degrees for the more restrictive az-
One way to realize this scheme is to rotate the image such imuth angle θ . This error range is discussed in the next sec-
that the light direction ω(φ lj ) is aligned with the y-axis and tion. Even for the zenith angle φ , only the second light in
the light on the left, see Figure 3. Then we simply scan each the Quilt scene returned a larger error due to the bouncing of
raster line i, starting from the silhouette boundary on left into that light in the surface on the left. Table 1 shows all the data
the interior. We detect the set of points {phi lo
i j } or {pi j }, cor- for the input images shown in Figure 4: for each light source
v
responding zenith angles {θi j } and luminances Li j . Thus for present in the scene, we show the real measured locations
the light j, the resultant zenith angle is the weighted sum: of the light sources, the results output by our algorithm and
the corresponding absolute error. The light probe used in the
∑i Livj θi j
θj = (2) first three images is the apple; for the other two, we used the
∑i Livj head of the guitar player and the Scottish quilt.

By using two objects in the image as light probes and re-

peating the process for the second one, we can approximate
the position of a point light source, e.g.: a source that is not
infinitely far away. Given that the directions computed for
both objects cannot be absolutely precise (we asssume direc-
tional lights), there will be no intersection point. We simply
place the light source halfway between the points d1 and d2
defining its minimum distance.
Once we have estimates of the light directions, estimat-
ing the relative intensities is fairly straight forward. For each
light j, we compute the total sum of luminances normalized
by the occlusion function of the light, over all the pixels i
of the contour of the object. The intensity of the light I j is
proportional to: Figura 4: Input images for the error analysis of Table 1.
I j ∝ ∑ Lvj /Ω(ni , ω j ) (3) From left to right: Apple1, Apple2 and Apple3, Guitar and
i Quilt.

Any potentially remaining light sources will be treated by

our algorithm as ambient illumination which we will explain Light 1 Light 2 Light 3
next. φ θ φ θ φ θ
−15.00 40.00 165.00 −40.00 − −
Guitar Apple3 Apple2 Apple1

R
A 5.71 35.31 162.25 −64.03 − −
3.4. Ambient Illumination E 20.71 4.69 2.75 24.03 − −
R 90.00 −70.00 − − − −
The shading contribution of the ambient light is assumed to
A 94.54 −65.70 − − − −
be constant for all pixels and we can therefore estimate its E 4.54 4.3 − − − −
intensity by analyzing pixels in the shadow regions. We al- R 180.00 0.00 0.00 0.00 − −
ready have detected the shadow lines in the previous step. A 168.50 14.48 0.0 11.31 − −
The region bounded by these shadow lines is determined to E 12.50 14.48 0.00 11.31 − −
be a shadow region. We average the set of samples along R 180.00 10.00 30.00 −45.00 260.00 45.00
these boundaries. This ambient intensity estimate is also rel- A 185.71 29.66 25.64 −49.19 272.29 41.48
ative to the previously detected lights. E 5.71 19.66 4.36 4.19 12.29 3.16
R 10.00 −35.00 120.00 −10.00 − −
Quilt

A 24.70 −51.79 162.25 4.74 − −

4. Results E 14.70 16.79 42.25 14.74 − −

We have tested our algorithm on several images with con- Tabla 1: Real measured light directions (R), value returned
trolled (known) light configurations, in order to measure by our algorithm (A) and absolute error (E) for the zenith φ
the errors of our light detection. The images include var- and azimuth θ angles in the scenes depicted in Figure 4.
ied configurations (see Figure 4): Apple1, Apple2 and Ap-
ple3 show a relatively simple geometry under very different
lighting schemes (with one or two light sources, plus am- We have further tested our algorithm on uncontrolled im-
bient light). The Guitar and Quilt images show much more ages, depicting scenes with unknown illuminations and vary-
complex scenes lit by three and two light sources respec- ing degrees of diffuse-directional lighting ratios. Given that
tively. The light directions returned by our algorithm show we obviously cannot provide error measures in those cases,

c The Eurographics Association 2009.
 Jorge Lopez-Moreno & Sunil Hadap & Erik Reinhard & Diego Gutierrez / Light source detection in photographs

we provide visual validation of the results by rendering a

synthetic object with the lighting scheme returned by our al-
gorithm. Figure 5, left, shows the original image and an un-
textured version of the 3D objects to be rendered. The image
on the right shows the results of illuminating the 3D objects
with the output returned by our algorithm. The chosen light
probe was one of the mushrooms. Figure 6 shows additional
examples of uncontrolled input images with synthetic ob-
jects rendered into them, using the head of the doll and the
whole human figure as light probes respectively. Note how Figura 7: Spheres rendered with information from the Gui-
our system is robust enough even if the light probe is com- tar image in Figure 4. Left: using the image as an environ-
posed of multiple objects with very different BRDFs (such ment map. Middle: using the real measured data. Right: us-
as the skin, glasses and hair in the doll image). Shadows cast ing the results of our algorithm. Our algorithm provides a
onto the original images are generated in the usual way by much better solution if the light sources are not present in
defining synthetic planes at approximately the right locations the original image.
when placing the synthetic objects.

one light and no knowledge of the actual 3D geometry is

required.
Additionally, this error (around 15 degrees in average)
might be below the human perception sensitivity as sug-
gested by the work of [OCS05] and shown in our rendered
images. However the accuracy of human vision in light de-
tection has not been measured yet and it should be quantified
Figura 5: Rendering synthetic objects into the images. Left, by psychophysical experiments in a future research.
top: original input image. Left, bottom: 3D models lit ac- We have shown good results both with controlled lighting
cording to the output of our light detection algorithm. Right: environments (where the light positions were measured and
final result, with the 3D models textured and inserted in the thus numerical data could be compared), and uncontrolled
image. settings (with free images downloaded from the internet,
and rendering synthetic objects with the results of our algo-
rithm). Our algorithm could potentially help photographers
mimic a given lighting scheme inspired by any other shot,
for which a reduced set of light directions (namely the ty-
pical three-light setup made up of key, fill and rim lights) is
preferable.
It could be argued that given that humans are not particu-
larly good at detecting light sources, simpler algorithms that
Figura 6: Additional examples of synthetic objects rendered approximate light sources could be employed as well. For
into images using the results of our algorithm. Left: synthetic instance, in the context of rendering synthetic objects into
teapot. Right: synthetic cone. existing images, one of the most popular recent approaches
is to build an environment map from the image. Whilst this
approach would provide reasonable results in certain cases
(as shown in [KRFB06]), it would fail if the main light
5. Discussion and Future Work sources are actually outside the image. One such example
would be the Guitar image in Figure 4. If we were to render
We have presented a novel light detection algorithm from
an object into the image, it would appear unrealistically dark.
single images, which only requires the silhouette of any ob-
Figure 7 shows a sphere rendered with the actual measured
ject in the image as additional user input. Our method yields
lights for that scene, compared to the results from rendering
a result in less than 4 seconds using a 512x512 version of
with an environment map and using the lights detected by
the original image. Although it works in smaller resolutions,
our algorithm.
higher resolutions have small effect on its accuracy. It may
seem that the average error of our method is too high in Several existing applications can benefit from our sys-
comparison with previous works in the field, however in op- tem, specifically those based on combining pictures from
position to those works we are not limited to detecting just an existing stack to create novel images. This kind of appli-

c The Eurographics Association 2009.
 Jorge Lopez-Moreno & Sunil Hadap & Erik Reinhard & Diego Gutierrez / Light source detection in photographs

cations are gaining popularity due to, among other factors, [LR89] L EE C., ROSENFELD A.: Improved methods of estimated
the current existence of huge databases and their accessibil- shape from shading using the light source coordinate system. In
ity through the internet. Some examples include Photo Clip Shape from shading, Horn B., Brooks M., (Eds.). MIT Press,
1989, pp. 323–569.
Art [LHE∗ 07], Interactive Digital Photomontage [ADA∗ 04]
or Photo Tourism [SSS06]. [MG97] M ARSCHNER S. R., G REENBERG D. P.: Inverse light-
ing for photography. In Fifth IST/SID Color Imaging Conference
We assume global convexity for the chosen de-facto light (1997), pp. 262–265.
probes in the images. Whilst this is true for most objects, [NE01] N ILLIUS P., E KLUNDH J.-O.: Automatic estimation of
the algorithm will return wrong values if a concave object is the projected light source direction. In CVPR (2001), pp. I:1076–
chosen instead. Our algorithm will also fail in the presence 1083.
of purely reflective or transparent (refractive) objects chosen [NN04] N ISHINO K., NAYAR S. K.: Eyes for relighting. ACM
as light probes, which break our assumption about shading. Transactions on Graphics (Proceedings of ACM SIGGRAPH) 23,
3 (2004), 704–711.
In these cases, an approach similar to [NN04] may be more
suitable, although previous knowledge about the geometry [OCDD01] O H B. M., C HEN M., D ORSEY J., D URAND F.:
Image-based modeling and photo editing. In SIGGRAPH ’01:
of the objects in the image would be needed. As future work, Proceedings of the 28th annual conference on Computer Graph-
we would like to address these cases. ics and Interactive Techniques (2001), 433-442, (Ed.).
[OCS05] O STROVSKY Y., C AVANAGH P., S INHA P.: Perceiv-
6. Acknowledgements ing illumination inconsistencies in scenes. Perception 34 (2005),
1301–1314.
This research was partially funded by a generous gift from [Pen82] P ENTLAND A.: Finding the illuminant direction. Journal
Adobe Systems Inc and and the Spanish Ministry of Science of the Optical Society of America A 72, 4 (1982), 448–455.
and Technology (TIN2007-63025). [PSG01] P OWELL M., S ARKAR S., G OLDGOF D.: A simple
strategy for calibrating the geometry of light sources. IEEE
Transactions on Pattern Analysis and Machine Intelligence 23,
References 9 (2001), 1022–1027.
[ADA∗ 04] AGARWALA A., D ONTCHEVA M., AGRAWALA M., [SSI99] S ATO I., S ATO Y., I KEUCHI K.: Illumination distribution
D RUCKER S., C OLBURN A., C URLESS B., S ALESIN D., C O - from brightness in shadows: Adaptive estimation of illumination
HEN M.: Interactive digital photomontage. ACM Transactions distribution with unknown reflectance properties in shadow re-
on Graphics 23, 3 (2004), 294–302. gions. In ICCV (2) (1999), pp. 875–882.
[BH85] B ROOKS M., H ORN B.: Shape and source from shading. [SSS06] S NAVELY N., S EITZ S. M., S ZELISKI R.: Photo
In Proc. Int. Joint Conf. Artificial Intell. (1985), pp. 932–936. tourism: Exploring photo collections in 3d. ACM Transactions
[GHH01] G IBSON S., H OWARD T., H UBBOLD R.: Flexi- on Graphics 25, 3 (2006), 835–846.
ble image-based photometric reconstruction using virtual light [VY94] V EGA E., YANG Y.-H.: Default shape theory: with the
sources. Computer Graphics Forum 19, 3 (2001), C203–C214. application to the computation of the direction of the light source.
[HA93] H OUGEN D., A HUJA N.: Estimation of the light source Journal of the Optical Society of America A 60 (1994), 285–299.
distribution and its use in integrated shape recovery from stereo [VZ04] VARMA M., Z ISSERMAN A.: Estimating illumination di-
shading. In ICCV (1993), pp. 29–34. rection from textured images. In Proceedings of the IEEE Con-
[Hor86] H ORN B.: Robot Vision. McGraw-Hill, 1986. ference on Computer Vision and Pattern Recognition, Washing-
[KP03] KOENDERINK J. J., P ONT S. C.: Irradiation direction ton, DC (June 2004), vol. 1, pp. 179–186.
from texture. Journal of the Optical Society of America 20, 10 [WAC07] WANG J., AGRAWALA M., C OHEN M.: Soft scissors:
(2003), 1875Ű–1882. An interactive tool for realtime high quality matting. ACM Trans-
[KRFB06] K HAN E. A., R EINHARD E., F LEMING R., actions on Graphics 26, 3 (2007), 9.
B ÜLTHOFF H.: Image-based material editing. ACM Transac- [WS02] WANG Y., S AMARAS D.: Estimation of multiple illu-
tions on Graphics (SIGGRAPH 2006) 25, 3 (2006), 654–663. minants from a single image of arbritary known geometry. In
[KvDP04] KOENDERINK J. J., VAN D OORN A. J., P ONT S. C.: ECCV02 (2002), vol. 3, pp. 272–288.
Light direction from shad(ow)ed random gaussian surfaces. Per- [YY91] YANG Y., Y UILLE A.: Sources from shading. In Com-
ception 33, 12 (2004), 1405–1420. puter Vision and Pattern Recognition (1991), pp. 534–539.
[LB01] L ANGER M. S., B ÜLTHOFF H. H.: A prior for global [ZK02] Z HOU W., K AMBHAMETTU C.: Estimation of illumi-
convexity in local shape-from-shading. Perception 30 (2001), nant direction and intensity of multiple light sources. In ECCV
403–410. ’02: Proceedings of the 7th European Conference on Computer
[LF06] L AGGER P., F UA P.: Using specularities to recover mul- Vision-Part IV (London, UK, 2002), Springer-Verlag, pp. 206–
tiple light sources in the presence of texture. In ICPR ’06: Pro- 220.
ceedings of the 18th International Conference on Pattern Recog- [ZTCS99] Z HANG R., T SAI P., C RYER J., S HAH M.: Shape
nition (Washington, DC, USA, 2006), IEEE Computer Society, from shading: A survey. IEEE Transactions on Pattern Analy-
pp. 587–590. sis and Machine Intelligence 28, 8 (1999), 690–706.
[LHE∗ 07] L ALONDE J.-F., H OIEM D., E FROS A. A., [ZY01] Z HANG Y., YANG Y.-H.: Multiple illuminant direction
ROTHER C., W INN J., C RIMINISI A.: Photo clip detection with application to image synthesis. IEEE Trans. Pat-
art. ACM Transactions on Graphics (SIGGRAPH 2007) tern Anal. Mach. Intell. 23, 8 (2001), 915–920.
26, 3 (August 2007). See the project webpage at
http://graphics.cs.cmu.edu/projects/photoclipart.

c The Eurographics Association 2009.