Computer Vision: Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm - 4:20pm

Computer Vision
Spring 2006 15-385,-685

Instructor: S. Narasimhan
Wean 5403
T-R 3:00pm 4:20pm
Lecture #12

Midterm March 9
Syllabus until and including Lightness and Retinex
Closed book, closed notes exam in class.
Time: 3:00pm 4:20pm
Midterm review class next Tuesday (March 7)
(Email me by March 6 specific questions)
If you have read the notes and readings, attended
all classes, done assignments well, it should be
a walk in the park

Mechanisms of Reflection
source
surface
reflection
surface
incident
direction
body
reflection
Body Reflection:
Diffuse Reflection
Matte Appearance
Non-Homogeneous Medium
Clay, paper, etc
Surface Reflection:
Specular Reflection
Glossy Appearance
Highlights
Dominant for Metals
Image Intensity = Body Reflection + Surface Reflection

Example Surfaces
Body Reflection:
Diffuse Reflection
Matte Appearance
Non-Homogeneous Medium
Clay, paper, etc
Surface Reflection:
Specular Reflection
Glossy Appearance
Highlights
Dominant for Metals
Many materials exhibit
both Reflections:

Diffuse Reflection and Lambertian BRDF
viewing
direction
surface
element
normal
incident
direction
i
n
v
s

d
r r i i
f ) , ; , (
Lambertian BRDF is simply a constant :
albedo
Surface appears equally bright from ALL directions! (independent of )
Surface Radiance :
v
Commonly used in Vision and Graphics!
s n I I L
d
i
d
. cos

source intensity
source intensity I

Diffuse Reflection and Lambertian BRDF

White-out: Snow and Overcast Skies
CANT perceive the shape of the snow covered terrain!
CAN perceive shape in regions
lit by the street lamp!!
WHY?

Diffuse Reflection from Uniform Sky
Assume Lambertian Surface with Albedo = 1 (no absorption)
Assume Sky radiance is constant
Substituting in above Equation:
Radiance of any patch is the same as Sky radiance !! (white-out condition)

2 /
0
sin cos ) , ; , ( ) , ( ) , (
i i i i r r i i i i
src
r r
surface
d d f L L
sky
i i
src
L L ) , (

1
) , ; , (
r r i i
f
sky
r r
surface
L L ) , (

Specular Reflection and Mirror BRDF
source intensity I
viewing
direction
surface
element
normal
incident
direction
n
v
s
r
specular/mirror
direction
) , (
i i

) , (
v v

) , (
r r

Mirror BRDF is simply a double-delta function :
Valid for very smooth surfaces.
All incident light energy reflected in a SINGLE direction (only when = ).
Surface Radiance :
) ( ) (
v i v i s
I L +
v r
) ( ) ( ) , ; , (
v i v i s v v i i
f +
specular albedo

Combing Specular and Diffuse: Dichromatic Reflection
Observed Image Color = a x Body Color + b x Specular Reflection Color
R
G
B
Klinker-Shafer-Kanade 1988
Color of Source
(Specular reflection)
Color of Surface
(Diffuse/Body Reflection)
Does not specify any specific model for
Diffuse/specular reflection

Diffuse and Specular Reflection
diffuse specular diffuse+specular

Photometric Stereo
Lecture #12

Image Intensity and 3D Geometry
Shading as a cue for shape reconstruction
What is the relation between intensity and shape?
Reflectance Map

Surface Normal
N
surface normal
y
z
x
Equation of plane 0 + + + D Cz By Ax
0 + + +
C
D
z y
C
B
x
C
A
or
Let
p
C
A
x
z

q
C
B
y
z

( ) 1 , , 1 , , q p
C
B
C
A
,
_
N
Surface normal

Surface Normal
X
Y
Z
X
Y
X p
Y q

Gradient Space
y
z
x
1 z
q
p
1
s
n
N
( )
1
1 , ,
2 2
+ +

q p
q p
N
N
n
( )
1
1 , ,
2 2
+ +

S S
S S
q p
q p
S
S
s
Normal vector
Source vector
i
( )
1 1
1
cos
2 2 2 2
+ + + +
+ +

S S
S S
i
q p q p
qq pp
s n
1 z
plane is called the Gradient Space (pq plane)
Every point on it corresponds to a particular surface orientation
S

Reflectance Map
Relates image irradiance I(x,y) to surface orientation (p,q)

for given source direction and surface reflectance
Lambertian case:
( ) y x I ,
s
v
n
i
: source brightness
: surface albedo (reflectance)
: constant (optical system)
k
c
Image irradiance:
s n kc kc I
i
cos
Let
1 kc
then s n
i
I cos

Lambertian case
( )
( ) q p R
q p q p
qq pp
I
S S
s s
i
,
1 1
1
cos
2 2 2 2

+ + + +
+ +
s n
Reflectance Map
(Lambertian)
cone of constant
i
Iso-brightness contour
Reflectance Map

Lambertian case
0 . 1
3 . 0
0 . 0
9 . 0
8 . 0
( ) 7 . 0 , q p R
p
q
90
i
( ) 0 1 + +
S S
qq pp
( )
S S
q p ,
iso-brightness
contour
Note: is maximum when ( ) q p R , ( ) ( )
S S
q p q p , ,
Reflectance Map

Glossy surfaces (Torrance-Sparrow reflectance model)

( ) ( ) q p R G p
kc
kc I
r
s
i
d
,
cos
cos +
diffuse term specular term

p
q
( ) 5 . 0 , q p R
( )
S S
q p ,
Diffuse peak
Specular peak
Reflectance Map

Shape from a Single Image?
Given a single image of an object with known surface

reflectance taken under a known light source, can we
recover the shape of the object?
Given R(p,q) ( (p
S
,q
S
) and surface reflectance) can we
determine (p,q) uniquely for each image point?
NO
p
q

Solution
Take more images
Photometric stereo
Add more constraints
Shape-from-shading (next class)

Photometric Stereo
p
q
( )
1 1
,
S S
q p
( )
2 2
,
S S
q p
( )
3 3
,
S S
q p

We can write this in matrix form:

Image irradiance:
,
_
kc
1 1
s n I
1
s
n
v
2
s
2 2
s n I
3
s
3 3
s n I
n
s
s
s
1
1
1
]
1
1
1
1
]
1
T
T
T
I
I
I
3
2
2
2
1
1
Lambertian case:
s n
i
kc I cos
Photometric Stereo

Solving the Equations
n
s
s
s
1
1
1
]
1
1
1
1
]
1
T
T
T
I
I
I
3
2
2
2
1
1
I S n
~
1 3 3 3 1 3
I S n
1
~

n
~

n
n
n
n
~
~
~

inverse

More than Three Light Sources
Get better results by using more lights

n
s
s
1
1
1
]
1
1
1
1
]
1
T
N
T
N
I
I

1
1
Least squares solution:

n S I
~
n S S I S
~
T T
( ) I S S S n
T T
1
~

Solve for as before n ,

Moore-Penrose pseudo inverse
( ) ( ) 1 3 3 1 N N

Color Images
The case of RGB images
get three sets of equations, one per color channel:
Simple solution: first solve for using one channel
Then substitute known into above equations to get
Or combine three channels and solve for

Sn I
G G

Sn I
B B

Sn I
R R

( )
B G R
, ,
n
n
Sn I I I I + +
2 2 2
B G R
n

Computing light source directions
Trick: place a chrome sphere in the scene

the location of the highlight tells you the source direction

For a perfect mirror, light is reflected about N

Specular Reflection - Recap
'
otherwise 0
if r v
i
e
R
R
We see a highlight when
Then is given as follows:

n
v
s
r v
s
( ) r n r n s 2
r
i

Computing the Light Source Direction
Can compute N by studying this figure

Hints:
use this equation:
can measure c, h, and r in the image
N
rN
C
H
c
h
Chrome sphere that has a highlight at position h in the image
image plane
sphere in 3D

Depth from Normals
Get a similar equation for V
2
Each normal gives us two linear constraints on z
compute z values by solving a matrix equation
V
1
V
2
N

Limitations
Big problems
Doesnt work for shiny things, semi-translucent things
Shadows, inter-reflections
Smaller problems
Camera and lights have to be distant
Calibration requirements
measure light source directions, intensities
camera response function

Trick for Handling Shadows
Weight each equation by the pixel brightness:
Gives weighted least-squares matrix equation:
Solve for as before

n
s
s
1
1
1
]
1
1
1
1
]
1
T
N N
T
N
I
I
I
I

1
1
2
2
1
n ,
( ) ( )
i i i i
I I I s n

Original Images

Results - Shape
Shallow reconstruction
(effect of interreflections)
Accurate reconstruction
(after removing interreflections)

Results - Albedo
No Shading Information

Original Images

Results - Shape

Results - Albedo

Results
1. Estimate light source directions
2. Compute surface normals
3. Compute albedo values
4. Estimate depth from surface normals
5. Relight the object (with original texture and uniform albedo)

Next Class
Shape from Shading
Reading: Horn, Chapter 11.

Computer Vision: Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm - 4:20pm

Uploaded by

Copyright:

Available Formats

Computer Vision: Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm - 4:20pm

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Computer Vision: Spring 2006 15-385,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm - 4:20pm

Uploaded by

Copyright:

Available Formats

Computer Vision

Spring 2006 15-385,-685

Shading as a cue for shape reconstruction

What is the relation between intensity and shape?

Relates image irradiance I(x,y) to surface orientation (p,q)

Glossy surfaces (Torrance-Sparrow reflectance model)

diffuse term specular term

Given a single image of an object with known surface

Take more images

Add more constraints

Shape-from-shading (next class)

We can write this in matrix form:

Get better results by using more lights

Least squares solution:

Solve for as before n ,

The case of RGB images

get three sets of equations, one per color channel:

Simple solution: first solve for using one channel

Then substitute known into above equations to get

Or combine three channels and solve for

Trick: place a chrome sphere in the scene

For a perfect mirror, light is reflected about N

We see a highlight when

Then is given as follows:

Can compute N by studying this figure

Doesnt work for shiny things, semi-translucent things

Camera and lights have to be distant

Weight each equation by the pixel brightness:

Gives weighted least-squares matrix equation:

Solve for as before

Shape from Shading

Reading: Horn, Chapter 11.

You might also like