JPEG and H.26x Standards

Lesson 5
JPEG and H.26x Standards
Video Data Size and Bit Rate

DCT Transform and Quantization
JPEG Standard for Still Image
Intra-frame and Inter-frame Compression
Block-based Motion Compensation
H.261 Standard for Video Compression
H.263, H.263+, H.263++, H.26L, H.264
1
Video Bit Rate Calculation

width ~ pixels (160, 320, 640, 720, 1280, 1920, )
height ~ pixels (120, 240, 480, 485, 720, 1080, )
depth ~ bits per pixel (1, 4, 8, 15, 16, 24, )
fps ~ frames per second (5, 15, 20, 24, 30, )
compression factor
(1 ~ 100 ~ )
time
Fn
width * height * depth * fps

compression factor
= bits/sec
bps
One Frame =
3 pictures (YCrCb)
F2
F1
Uncompressed Video Data Size

compression factor = 1
Size of uncompressed video in gigabytes

1 sec
1 min
1 hour
1000 hours
1920x1080
0.19
11.20
671.85
671,846.40
1280x720
0.08
4.98
298.60
298,598.40
640x480
0.03
1.66
99.53
99,532.80
320x240
0.01
0.41
24.88
24,883.20
160x120
0.00
0.10
6.22
6,220.80
Image size of video
1280x720 (1.77)
640x480 (1.33)
320x240
160x120
Effects of Compression
storage for 1 hour of compressed video in megabytes
Compression
ration
1:1
3:1
6:1
25:1
100:1
1920x1080
671,846
223,949
111,974
26,874
6,718
1280x720
298,598
99,533
49,766
11,944
2,986
640x480
99,533
33,178
16,589
3,981
995
320x240
24,883
8,294
4,147
995
249
160x120
6,221
2,074
1,037
249
62
3 bytes/pixel, 30 frames/sec
Coding Overview
Digitize
640x480
Subsample to reduce data
Compression algorithms exploit:
Spatial redundancy - correlation between neighboring pixels
320x240
Intra-frame compression
remove redundancy within frame
Temporal redundancy - correlation betw. frames

Inter-frame compression
Remove redundancy between frames
Inter-frames
Symbol Coding
Efficient coding of sequence of symbols
Intra-frame
RLC (Run Length Coding)

Huffman coding
5
Transform Coding
N x M image
An image conversion process that transforms an image

from the spatial domain to the frequency domain.
Subdivide an individual N x M image into small n x n blocks
Each n x n block undergoes a reversible transformation
Basic approach:
De-correlate the original block - radiant energy is redistributed
amongst only a small number of transform coefficients
Discard many of the low energy coefficients (through quantization)
f(i,j)
i
j
F(u,v)
Transform
Function
nxn
blocks
YCrCb
Fq(u,v)
u
Quantizer
q(u,v)
Quanti. Table
v
6
DCT nxn Discrete Cosine Transform

F=Dx f
4C(u)C(v)
F[u,v] =
n2
F, D, f are n-by-n matrixes
n-1 n-1
f(j,k) cos
(2j+1)up
2n
j=0 k=0
where
C(w) =
IDCT is very similar

8x8 DCT
coefficients
2
1
0.7
1
0.9
0.8
0.7
0.6
0.4
0.2
cos
(2k+1)vp
2n
for w=0
for w=1,2,,n-1
0.7 0.7 0.7 0.7 0.7 0.7 0.7
0.8 0.6 0.2 -0.2 -0.6 0.8
-1
0.4 -0.4 -0.9 -0.9 -0.4 0.4 0.9
-0.2
-1 -0.6 0.6
1 0.2 -0.8
-0.7 -0.7 0.7 0.7 -0.7 -0.7 0.7
-1 0.2 0.8 -0.8 -0.2
1 -0.6
-0.9 0.9 -0.4 -0.4 0.9 -0.9 7 0.4
-0.6 0.8
-1
1 -0.8 0.6 -0.2
Quantization
Purpose of quantization
Achieve high compression by representing DCT coefficients with no
greater precision than necessary
Discard information which is not visually significant
After output from the FDCT, each of the 64 DCT coefficients is quantized
Many-to-one-mapping => fundamentally lossy process
Fq[u,v] = Round ( F[u,v] / q[u,v])
Example: F[u,v] =101101 = 45 (6 bits).
If q[u,v] = 4, truncate to 4 bits, Fq[u,v] =1011
Example:
2x2 block
F[u,v] =
45
8
12
3
Q[u,v] =
4
6
6
8
Fq[u,v] =
11
1
2
0
Quantization is the principal source of lossiness in DCT-based encoders

Uniform quantization: each F[u,v] is divided by the same constant N
Non-uniform quantization: use quantization tables from psycovisual
experiments to exploit the limit of human visual system
8
DCT and Quantization Example

DC component, others called AC
Fq
F -1
f -1
JPEG Image Compression Standard

Mainly for still image (gray and color)
Four Modes:
- Lossless JPEG
- Sequential (Baseline) JPEG
- Progressive JPEG
- Hierarchical JPEG
Hybrid Coding Techniques:

- DCT Coding
- Run Length Encoding(RLE)
- Huffman Coding
- Linear Prediction (only in lossless mode)
New Standard: JPEG2000

Motion JPEG for video
10
Overview of Baseline JPEG

YCrCb
.jpeg file
11
Block Transform Encoding

DCT
Zig-zag ordering
Quantize
011010001011101...
Run-length
Code
Huffman
Code
12
Quantize Table
15 10 10 16
10 16 12 16
10 8 16 16
Example of Block Encoding

DC component
139
144
150
159
144
151
155
161
149
153
160
162
153
156
163
160
DCT
1260 -1 -12
-23 -17 -6
-11 -9 -2
-7
-2
0
-5
-3
2
1
16 12 16 32
Quantize
original image
AC components
79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
run-length
code
0
1
0
0
0
2
0
79
-2
-1
-1
-1
-1
0
Huffman
code
zigzag
79
-2
-1
0
0
-1
-1
0
-1
0
0
0
10011011100011...
coded bitstream < 10 bits (0.55 bits/pixel)

13
0
0
0
0
Result of Coding/Decoding
139
144
150
159
144
151
155
161
149
153
160
162
144
156
155
160
153
156
163
160
146
150
156
161
149
152
157
161
152
154
158
162
reconstructed block
original block
-5
-4
-5
-1
-2
1
-1
0
0
1
3
1
1
2
5
-2
Small Loss
Neglect-able
errors
14
Examples
Uncompressed
(262 KB)
Compressed (50)
(22 KB, 12:1)
Compressed (1)
(6 KB, 43:1)
8 bits/pixel
0.67 bit/pixel
0.17 bit/pixel
15
JPEG vs. GIF

JPEG Advantages
more colors (GIF limited to 256)
lossless option
best for scanned photographs
progressive JPEG downloads rough image before whole
image arrives
GIF Advantages
transparent color setting
animated GIFs
better for flat color fields: clip art, cartoons, etc.
interlaced delivery downloads low resolution image before
whole image arrives
16
Intra- vs. Inter-frame Compression

Intra-frame compression
For still image like JPEG
Exploit the redundancy in
image (spatial redundancy)
Can be applied to individual
frames in a video sequence
Techniques
Subsampling (small size)
Block transform coding
Coarse quantization
Intra + inter-frame compression

For video like H.26x & MPEG
Exploit the similarities between
successive frames (temporal
redundancy)
Techniques
Subsampling (small frame rate)
Difference coding
Block-based difference coding
Block-based motion
compensation
Intra-frame
Inter-frames
17
Difference Coding
Compare pixels with previous frame
Only pixels that have been changed are updated
A fraction of the number of pixel values will be recorded
Overhead associated with which pixels are updated: what

if a large number of pixels are changed ?
Pixels values are slightly different even with no movement
18
of objects: ignore small changes (lossy)
Block-based Difference Coding
Difference coding at the block level
Send sequence of blocks rather than frames

If previous block similar, skip it or send difference
Update a whole block of pixels at once
160 x 120 pixels (19200 pixels) => 8x8 blocks (300 blocks)
Possible artifact at the border of blocks

Limitations of difference coding
Useless where there is a lot of motion (few pixels unchanged)
What if a camera itself is moving ?
Need to compensate for object motion

19

Motion compensation assumes that current frame can
be modeled as a translation of a previous frame
Search around block in previous frame for a better
matching block and encode position and error difference
20

Current frame is divided into uniform non-overlapping blocks
Each block in the current frame is compared to areas of similar size
from the preceding frame in order to find an area that is similar
The relative difference in locations is known as the motion vector
Because fewer bits are required to code a motion vector than to code
actual blocks, compression is achieved.
motion vector
21
Bidirectional Motion Compensation

future
Bidirectional motion compensation

present
Areas just uncovered are
not predictable from the past,
past
but can be predicted from the future
Search in both past and future frames
Effect of noise and errors can be reduced by averaging
between previous and future frames
Bi-directional interpolation provides a high degree of
compression
Requires that frames be encoded and transmitted in a
different order from which they will be displayed.
In reality, exact matching is not possible, thus lossy
compression
22
Overview of H.261
Developed by CCITT (Consultative Committee for
International Telephone and Telegraph) in 1988-1990
Designed for videoconferencing, video-telephone
applications over ISDN telephone lines.
Bit-rate is p x 64 Kbps, where p ranges from 1 to 30
(2048 kbps)
Supports CCIR 601 CIF (352 x 288) and QCIF (176 x
144) images with 4:2:0 subsampling.
Significant influence on H.263, MPEG 1-4, etc.
23
Frame Sequence of H.261

Two frame types: Intra-frames (I-frames) and Inter-frames
(P-frames): I-frame provides an accessing point, it uses
basically JPEG.
P-frames use "pseudo-differences" from previous frame
("predicted"), so frames depend on each other.
24
Intra-frame Coding
Macroblock:
16 x 16 pixel areas
on Y plane of original
image.
Usually consists of 4
Y blocks, 1 Cr block,
and 1 Cb block
(4:2:0 or 4:1:1)
Quantization is by
constant value for all
DCT coefficients (i.e., no
quantization table as in
JPEG).
25
Inter-frame Coding
26
Motion Vector Searches

C(x+k, y+l): macro block pixels in the target
R(x+i+k, y+j+l): macro block pixels in the
reference
MAE (i, j )
1
N 1
N 1
C ( x k , y l ) R( x i k , y j l )
2 k 0 l 0
N
The goal is to find a vector (u, v) such that
the mean Absolute Error, MAE(u, v)
is minimum:
1. Full Search Method
2. Two-dimensional Logarithmic Search
3. Hierarchical Motion Estimation
27
Encoder
28
H.262, H.263 and H.264

H.262 = MPEG-2 jointly by ITU and ISO/IEC
ITU-T Rec. H.263 v1 (1995)
Current best standard for practical video telecommunication
Has overtaken H.261 as videoconferencing codec
Superior to H.261 at all bit rates (1/2)
Video size: Sub-QCIF (128x96), QCIF (176x144),
CIF(352x288), 4CIF(704X576), 16CIF (1408x1152)
PB frames mode (bidirectional prediction)
4 motion vector for each block, pixel accuracy
Arithmetic coding efficient than Huffman coding in H.261
H.263 v2 (H.263+, 1997)
H.263 v3 (H.263++, 2000), H.26L (2002)
H.264/AVC (now)
29
Demos of Image GIF and JPEG Coding

JPEG and H.26x Standards

Uploaded by

Copyright:

Available Formats

JPEG and H.26x Standards

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

JPEG and H.26x Standards

Uploaded by

Copyright:

Available Formats

Lesson 5

JPEG and H.26x Standards

Video Data Size and Bit Rate

Video Bit Rate Calculation

width * height * depth * fps

Uncompressed Video Data Size

Size of uncompressed video in gigabytes

Image size of video

Temporal redundancy - correlation betw. frames

RLC (Run Length Coding)

An image conversion process that transforms an image

DCT nxn Discrete Cosine Transform

F, D, f are n-by-n matrixes

IDCT is very similar

Quantization is the principal source of lossiness in DCT-based encoders

DCT and Quantization Example

JPEG Image Compression Standard

Hybrid Coding Techniques:

New Standard: JPEG2000

Overview of Baseline JPEG

Block Transform Encoding

Example of Block Encoding

coded bitstream < 10 bits (0.55 bits/pixel)

JPEG vs. GIF

Intra- vs. Inter-frame Compression

Intra + inter-frame compression

Overhead associated with which pixels are updated: what

Block-based Difference Coding

Difference coding at the block level

Send sequence of blocks rather than frames

Possible artifact at the border of blocks

Need to compensate for object motion

Block-based Motion Compensation

Block-based Motion Compensation

Bidirectional Motion Compensation

Bidirectional motion compensation

Frame Sequence of H.261

Motion Vector Searches

H.262, H.263 and H.264

Demos of Image GIF and JPEG Coding

You might also like