Scratch Detection & Restoration in Scratch Detection & Restoration in Old Videos & Still Images

Download as pdf or txt
Download as pdf or txt
You are on page 1of 39

Scratch Detection & Restoration in

Old Videos & Still Images

SUPERVISOR: DR. SIMANT DUBE & DR. AMITABHA MUKHERJEE

BY
NANDAN DUBEY(Y6279)
VISHAL AGRAWAL(Y6541)
SARIKA MOHAPATRA (Y6430)
.
Scratch Detection & Restoration
 Why important?
 Photographic archives are being transferred to digital
form
 Involves scanning of transmissive and reflective media,
namely slides or negatives, and prints
 Inevitably introduces new artifacts such as dust particles
and scratches => degradation of quality
 Why automate?
 Because of the huge number of the involved frames
Detection & Removal
 The problem of automatic defect removal is usually divided
into two stages: detection and removal
 Detection: Defective pixels identified
 Reconstruction: Defective pixels are replaced with estimated
original pixels
Hurdles?
 Separating true defects and intrinsic image features, such as
edges, line segments and various textures. Inaccurate
detection
 Inaccurate detection leads to distortion of intrinsic image
features during the reconstruction stage
Artifacts in Old Films
 The main defect in old films are dust spots, line
scratches, abrasion, blotch, film unsteadiness and
intensity flicker
 Line scratches are one of the most annoying artifacts in
old films
 Caused by particle caught in film transport mechanism,
and when film passes over the particles material being
abraded and line scratches are formed.
All about Scratch
 The width of line scratches are between 2-9 pixels
 In the center of line scratches, the intensity of the pixels
shows a very little fluctuation
 The level of the intensity values for line scratches is
higher than that of image edges
 Typically they are vertical (angle to the vertical upto 5
degrees)
 Loss of color information in the region is complete
(usually have bright or dark color)
All about Scratch
 Line scratch usually remains in a similar positions in
the consecutive frames
 What others assume is also scratch is perfectly vertical
and is in whole frame which is not the case
Scratch Detection
 Two major types:
 Spatial methods
 Temporal methods
 Spatial - Spatial detection methods uses strong
vertical scratch property
 Temporal - Tracking of scratch in motion
compensated image frames
OWE-based line scratch detection
 OWE is an undecimated wavelet transform with translation
invariant
 It locates the position of line scratches in each scale as same
as the position in the spatial domain without any information
damaged during the wavelet transform
 From this point of view, OWE is more appropriate than
decimated wavelet transforms in image post- processing
 Allows us to design a more efficacious and precise position
for scratch than traditional methods
Proposed Method
 Generalized Wavelet Expansion - based Algorithm
 To locate the line artifact and represent them precisely, the
algorithm adopts Over - complete Wavelet Expansion
(OWE) representation of original image
 At the edge of line scratches area, horizontal intensity
variation is much larger than vertical intensity variation
 We are using ‘Haar wavelet transform’ - fits well with the
sine model for scratch proposed by D. Tegolo[7]
Proposed Method
Steps involved:
 Convert the image to grey scale if it is RGB and change
the values in double to apply Wavelet transform in next
step.
 Take Over - complete wavelet expansion (OWE).
 Assume that the original image has m by n pixels, and
denote W v(x,y) as first scale vertical wavelet
decomposition coefficient at the position of (x, y), then
the sum is:
Proposed Method
 We find the local maxima and minima and check if there is
local maxima and minima in nearby pixel based on scratch
width(5 pixel both side).
 Based on the fact that there are sharp horizontal variations at
both sides of line scratches and local maximum and local
minimum will be found at the left side and right side of line
scratches respectively[4].
Proposed Method
 We can even tell the color of scratch.

White scratch
.
Threshold Locally
 For actual position of scratches
 local thresholding (50 cross 9 block )
 height and width property of scratch

 Our threshold is thus independent of image property


while others use predefined thresholds depending on
image properties
Better than others
 Used only universal line property of scratches (using
vertical scale wavelet decomposition)
 Independent of image properties
 Local thresholding
 Detected pixels only in the defect region and not across
complete height
Flowchart of Algorithm
Grey Scale
Colored
Image
Image

rgb2gray

Im2double
OWE(A,H,V,D)
Double values
Sum V
Mask (those i’s ) (Vertical)

R (i) * R (j) < 0 Local V


Scratch maxima (R)

V must have at least 35 of 50 as above


average
Restoration
 Restoration improves the subjective visual quality of
archive
 To restore the film without causing distortion to areas of
the frames that are not affected, the location of scratch
must be located precisely
 Moreover a fully automatic restoration system is very
hard to develop, because the correction of misalignment
of brightness, detection and removal of shot and scratch,
noise reduction, etc.
Restoration
 Assumption: Regions affected by scratches do not
contain original information
 Solution: Propagate neighboring clean information into
the degraded area
 The neighboring information can be found in the same
frame or also in the preceding and successive frame
exploiting the temporal coherency
Restoration Method
 Most algorithms based on the interpolation of the
damaged area from its neighboring part
 Restoration using polynomial, Fourier interpolation, 3-D
Autoregressive algorithm, MRF model and Bayesian
restoration etc.
 They consume too much computational time. Our
method intends to simplify the complexity of the model
for the scratch detection and to implement a system able
to detect and remove only line scratches in each frame
Restoration Method
 Median filtering based missing data detection
Functional block diagram
Digital
Analogue
Degraded degraded
Sequence/img sequence/img

Digital Encoder

Scratch
Detector OWE mask

Comparing
Interpolation frames ( Scratches
Only for (Lessen False
videos) detection )

Restored
Seq/image
Result Images scratch removed
.
Discussion and Conclusion
 THERE ARE two factors to be considered in discussing the
performance of this method. First of all, given some
scratched image or video frame how accurate is the detection
?
 Second, after detecting scratches what is the quality of
reconstruction ?
 The fact that the median filter reconstructs the texture in this
region well is more due to the fact that it rearranges existing
surrounding samples and conserves the randomness of the
background texture.
Discussion and Conclusion
 Visual results are not very good for spatially removed scratch
in our method. For temporal removal the ML3Dex results in
quality interpolation but it depends heavily on the integrity
of the motion estimates. The multi median filter fails
however, when the motion estimate is not sufficiently
accurate or the structures to be interpolated are more
complicated. We are still working on improving the
restoration algorithm. Most probably switch to ? .
Improvements
 SPATIAL SCRATCH detection methods can give various false
scratch alarms if they are adjusted to detect all scratches, while
they may miss some scratches if they are tuned to less precise
detection. It is clear that only spatial properties of an image frame
are insufficient for accurate scratch detection same is also true for
methods based on only temporal properties.
 We will try to propose a model which combine both. Interframe
temporal correlation is employed to verify candidate scratches that
are obtained from the spatial detection method proposed.
References
 [1] Jin Xu, Jinghuo Guan, Xingdong Wang,Jun Sun, Guangtao ZhaiAn OWE-based Algorithm
for Line Scratches Restoration in Old Movies, 2007
 [10] Efficient Optimization of Inpainting Scheme and Line Scratch Detection for Old Film
Restoration Seong-Whan Kim and Ki-Hong Ko[2006]
 [2] V. Bruni, D. Vitualano, A.C. Kokaram,”Fast removal of line scratches in old movies,
Proceedings of the 17th International Conference on Pat- tern Recognition, 2004, ICPR
2004, vol. 4, pp.827830, Augest 2004.
 [4] V. Bruni, D. Vitulano, A generalized model for scratch detection, IEEE Transactions on
Image Processing, vol. 13, pp. 4450, January 2004.
 [3] A. C. Kokaram, Detection and removal of line scratches in degraded mo- tion picture
sequences, Signal Processing VIII, vol. 1, pp. 58, September 1996.
 [5] M. Kemal Guillui, Oguzhan Urhan, Sarp Erttirk, Scratch Detection via Temporal
Coherency Analysis and Removal using Edge Priority Based Interpolation.
 [6] Joyeux, L.; Buisson, O.; Besserer, B.; Boukir, S., Detection and removal of line scratches in
motion picture films
 [7] D. tegolo, A genetic algorithm for scratch removal in static images.
 [8] Laurent Joyeux, Samia Boukir, Bernard Besserer, Film Line Scratch Removal Using
Kalman Filtering and Bayesian Restoration.
 [9] A.C.Kokaram, Interpolation of missing data in image sequences
THANK YOU

You might also like