Scratch Detection & Restoration in Scratch Detection & Restoration in Old Videos & Still Images
Scratch Detection & Restoration in Scratch Detection & Restoration in Old Videos & Still Images
Scratch Detection & Restoration in Scratch Detection & Restoration in Old Videos & Still Images
BY
NANDAN DUBEY(Y6279)
VISHAL AGRAWAL(Y6541)
SARIKA MOHAPATRA (Y6430)
.
Scratch Detection & Restoration
Why important?
Photographic archives are being transferred to digital
form
Involves scanning of transmissive and reflective media,
namely slides or negatives, and prints
Inevitably introduces new artifacts such as dust particles
and scratches => degradation of quality
Why automate?
Because of the huge number of the involved frames
Detection & Removal
The problem of automatic defect removal is usually divided
into two stages: detection and removal
Detection: Defective pixels identified
Reconstruction: Defective pixels are replaced with estimated
original pixels
Hurdles?
Separating true defects and intrinsic image features, such as
edges, line segments and various textures. Inaccurate
detection
Inaccurate detection leads to distortion of intrinsic image
features during the reconstruction stage
Artifacts in Old Films
The main defect in old films are dust spots, line
scratches, abrasion, blotch, film unsteadiness and
intensity flicker
Line scratches are one of the most annoying artifacts in
old films
Caused by particle caught in film transport mechanism,
and when film passes over the particles material being
abraded and line scratches are formed.
All about Scratch
The width of line scratches are between 2-9 pixels
In the center of line scratches, the intensity of the pixels
shows a very little fluctuation
The level of the intensity values for line scratches is
higher than that of image edges
Typically they are vertical (angle to the vertical upto 5
degrees)
Loss of color information in the region is complete
(usually have bright or dark color)
All about Scratch
Line scratch usually remains in a similar positions in
the consecutive frames
What others assume is also scratch is perfectly vertical
and is in whole frame which is not the case
Scratch Detection
Two major types:
Spatial methods
Temporal methods
Spatial - Spatial detection methods uses strong
vertical scratch property
Temporal - Tracking of scratch in motion
compensated image frames
OWE-based line scratch detection
OWE is an undecimated wavelet transform with translation
invariant
It locates the position of line scratches in each scale as same
as the position in the spatial domain without any information
damaged during the wavelet transform
From this point of view, OWE is more appropriate than
decimated wavelet transforms in image post- processing
Allows us to design a more efficacious and precise position
for scratch than traditional methods
Proposed Method
Generalized Wavelet Expansion - based Algorithm
To locate the line artifact and represent them precisely, the
algorithm adopts Over - complete Wavelet Expansion
(OWE) representation of original image
At the edge of line scratches area, horizontal intensity
variation is much larger than vertical intensity variation
We are using ‘Haar wavelet transform’ - fits well with the
sine model for scratch proposed by D. Tegolo[7]
Proposed Method
Steps involved:
Convert the image to grey scale if it is RGB and change
the values in double to apply Wavelet transform in next
step.
Take Over - complete wavelet expansion (OWE).
Assume that the original image has m by n pixels, and
denote W v(x,y) as first scale vertical wavelet
decomposition coefficient at the position of (x, y), then
the sum is:
Proposed Method
We find the local maxima and minima and check if there is
local maxima and minima in nearby pixel based on scratch
width(5 pixel both side).
Based on the fact that there are sharp horizontal variations at
both sides of line scratches and local maximum and local
minimum will be found at the left side and right side of line
scratches respectively[4].
Proposed Method
We can even tell the color of scratch.
White scratch
.
Threshold Locally
For actual position of scratches
local thresholding (50 cross 9 block )
height and width property of scratch
rgb2gray
Im2double
OWE(A,H,V,D)
Double values
Sum V
Mask (those i’s ) (Vertical)
Digital Encoder
Scratch
Detector OWE mask
Comparing
Interpolation frames ( Scratches
Only for (Lessen False
videos) detection )
Restored
Seq/image
Result Images scratch removed
.
Discussion and Conclusion
THERE ARE two factors to be considered in discussing the
performance of this method. First of all, given some
scratched image or video frame how accurate is the detection
?
Second, after detecting scratches what is the quality of
reconstruction ?
The fact that the median filter reconstructs the texture in this
region well is more due to the fact that it rearranges existing
surrounding samples and conserves the randomness of the
background texture.
Discussion and Conclusion
Visual results are not very good for spatially removed scratch
in our method. For temporal removal the ML3Dex results in
quality interpolation but it depends heavily on the integrity
of the motion estimates. The multi median filter fails
however, when the motion estimate is not sufficiently
accurate or the structures to be interpolated are more
complicated. We are still working on improving the
restoration algorithm. Most probably switch to ? .
Improvements
SPATIAL SCRATCH detection methods can give various false
scratch alarms if they are adjusted to detect all scratches, while
they may miss some scratches if they are tuned to less precise
detection. It is clear that only spatial properties of an image frame
are insufficient for accurate scratch detection same is also true for
methods based on only temporal properties.
We will try to propose a model which combine both. Interframe
temporal correlation is employed to verify candidate scratches that
are obtained from the spatial detection method proposed.
References
[1] Jin Xu, Jinghuo Guan, Xingdong Wang,Jun Sun, Guangtao ZhaiAn OWE-based Algorithm
for Line Scratches Restoration in Old Movies, 2007
[10] Efficient Optimization of Inpainting Scheme and Line Scratch Detection for Old Film
Restoration Seong-Whan Kim and Ki-Hong Ko[2006]
[2] V. Bruni, D. Vitualano, A.C. Kokaram,”Fast removal of line scratches in old movies,
Proceedings of the 17th International Conference on Pat- tern Recognition, 2004, ICPR
2004, vol. 4, pp.827830, Augest 2004.
[4] V. Bruni, D. Vitulano, A generalized model for scratch detection, IEEE Transactions on
Image Processing, vol. 13, pp. 4450, January 2004.
[3] A. C. Kokaram, Detection and removal of line scratches in degraded mo- tion picture
sequences, Signal Processing VIII, vol. 1, pp. 58, September 1996.
[5] M. Kemal Guillui, Oguzhan Urhan, Sarp Erttirk, Scratch Detection via Temporal
Coherency Analysis and Removal using Edge Priority Based Interpolation.
[6] Joyeux, L.; Buisson, O.; Besserer, B.; Boukir, S., Detection and removal of line scratches in
motion picture films
[7] D. tegolo, A genetic algorithm for scratch removal in static images.
[8] Laurent Joyeux, Samia Boukir, Bernard Besserer, Film Line Scratch Removal Using
Kalman Filtering and Bayesian Restoration.
[9] A.C.Kokaram, Interpolation of missing data in image sequences
THANK YOU