US 2004O234143A1

(19) United States

(12) Patent Application Publication (10) Pub. No.: US 2004/0234143 A1
Hagai et al. (43) Pub. Date: Nov. 25, 2004
(54) IMAGE ENCODING METHOD AND Publication Classification
PICTURE DECODING METHOD
(51) Int. Cl." ....................................................... G06K 9/36
(76) Inventors: Makoto Hagai, Moriguchi-shi (JP); (52) U.S. Cl. ............................................ 382/238; 382/236
Shinya Kadono, Nishinomiya-shi (JP);
Satoshi Kondo, Yawata-shi (JP);
Kiyofumi Abe, Kadoma-shi (JP) (57) ABSTRACT
Correspondence Address: The picture coding method comprises: a Second reference
WENDEROTH, LIND & PONACK, L.L.P. picture Selection Step of Selecting, as a Second reference
2033 K STREET N. W.
picture, any one of coded pictures located after the current
Sieron, DC 20006-1021 (US) picture to be coded in display order (S100); a predictive
image generation Step of Specifying a block in the first
(21) Appl. No.: 10/487,533 reference picture and a block in the Second reference picture,
and generating a predictive image from Said two blockS
(22) PCT Filed: Jun. 9, 2003 through pixel interpolation (S102 and S104); a coding step
of coding a difference between the predictive image and an
(86) PCT No.: PCT/JP03/07242 image of a current block to be coded (S106 and S108); and
(30) Foreign Application Priority Data an output Step of outputting Specification information for
Specifying the Second reference picture, together with a
Jul. 2, 2002 (JP)...................................... 2002-193027 result of the coding performed in the coding Step.

Variable length
coding unit

A prediction
rediction
type
unit
selectio
Motion
estimation 105
unit o iReP 1. O6
Pixel interpolation
Unit
Direct mode
processing unit
Patent Application Publication Nov. 25, 2004 Sheet 1 of 38 US 2004/0234143 A1

g
alS.
Patent Application Publication Nov. 25, 2004 Sheet 2 of 38 US 2004/0234143 A1

FIG. 2

P(i) = A-X1(i) + B-X2(i) + C

Patent Application Publication Nov. 25, 2004 Sheet 3 of 38 US 2004/0234143 A1
Patent Application Publication Nov. 25, 2004 Sheet 4 of 38 US 2004/0234143 A1

y
Chl
Patent Application Publication Nov. 25, 2004 Sheet 5 of 38 US 2004/0234143 A1
Patent Application Publication Nov. 25, 2004 Sheet 6 of 38 US 2004/0234143 A1

V
A.
dy y
CD \
E N
P CY

O
(N
H
CY
(N
Cy
H

VO O
?y
CD H

O
v
Y
Patent Application Publication Nov. 25, 2004 Sheet 7 of 38 US 2004/0234143 A1

806

ZAW‘T ZIH’TH
dS
O
Patent Application Publication Nov. 259 2004 Sheet 8 of 38 US 2004/0234143 A1

II
ZI
£I

-|–s—,
pu0 S
Patent Application Publication Nov. 25, 2004 Sheet 9 of 38 US 2004/0234143 A1

IZAWNS’'ZIHJa
Patent Application Publication Nov. 25, 2004 Sheet 10 of 38 US 2004/0234143 A1

GÐ
la

ZAW’T ZIH’TIH
dS
O
£II
Patent Application Publication Nov. 259 2004 Sheet 11 Of 38 US 2004/0234143 A1

09.

II'5ÐI
Patent Application Publication Nov. 25, 2004 Sheet 12 of 38 US 2004/0234143 A1

FIG. 12

Picture with smallest second reference index S100

Out of pictures located later in display order
I
Second reference picture

S102
Specify scaling vector and first
reference picture from second
reference picture

S104
Generate predictive image
in direct mode
S106
Generate prediction error
S108

End
Patent Application Publication Nov. 25, 2004 Sheet 13 of 38
Patent Application Publication Nov. 25, 2004 Sheet 14 of 38 US 2004/0234143 A1

s S

a.
Patent Application Publication Nov. 25, 2004 Sheet 15 of 38 US 2004/0234143 A1

=ZAWS'-TAWS
‘ZTIH-JI

Z’GT5I)- 69
I
Patent Application Publication Nov. 25, 2004 Sheet 16 of 38 US 2004/0234143 A1

FIG. 16

Picture with smallest second reference index S150

out of pictures located later in display order
Second reference picture

S152
Specify scaling vector and first
reference picture from second
reference picture

S154
Generate predictive image
in direct mode
S156
Predictive image + residual

End
Patent Application Publication Nov. 25, 2004 Sheet 17 of 38 US 2004/0234143 A1

ISg
U|

LdGÐ

80I

£(Z0EO?ÐIT' dS
O
TIH’Awu
Patent Application Publication Nov. 25, 2004 Sheet 18 of 38 US 2004/0234143 A1

ua peÐH
Patent Application Publication Nov. 25, 2004 Sheet 19 of 38 US 2004/0234143 A1

FIG. 19

Any one of pictures located later in display order S200

Second reference picture
S2O2
Specify scaling vector and first
reference picture from second
reference picture

S2O4
Generate predictive image
in direct mode
S2O6
Generate prediction error
S2O8
Prediction error
Remapping information Code
Second reference index rRI2
Patent Application Publication Nov. 25, 2004 Sheet 20 of 38 US 2004/0234143 A1

ZGI ZAW’T ZIH’TIH ZAWS'T ZI?u’TH-I

Patent Application Publication Nov. 25, 2004 Sheet 21 of 38 US 2004/0234143 A1

FIG 21

S250
Remapping information Obtai
Second reference index al

S252
Specify scaling vector and first
reference picture from second
reference picture Corresponding
to second reference index

S254
Generate predictive image
in direct mode
S256
Predictive image + residual
End
Patent Application Publication Nov. 25, 2004 Sheet 22 of 38 US 2004/0234143 A1
Patent Application Publication Nov. 25, 2004 Sheet 23 of 38 US 2004/0234143 A1

s|a: ø |£d
|(TZELF?,Ku–6oe)
|—„. lia?u (q) (O) (p)
Patent Application Publication Nov. 25, 2004 Sheet 24 of 38 US 2004/0234143 A1

FIG. 24

S300
Any scene change?

S3O4
Remapping
o Assign first reference No remapping
index of O to reference
picture in long-term memory
O Assign second reference index
of 0 to reference picture
subsequent to Current
picture to be coded

S306
Generate predictive image
S3O8
Generate prediction error
S310
Coding
Patent Application Publication Nov. 25, 2004 Sheet 25 of 38 US 2004/0234143 A1

ZSOE d
O

ZAW’T ZIH’TIH
Patent Application Publication Nov. 25, 2004 Sheet 26 of 38 US 2004/0234143 A1

q?ue 6une.W

ZAW’TAWZIH'IH ZAWS'T ZIYJu‘H-I

Patent Application Publication Nov. 25, 2004 Sheet 27 of 38 US 2004/0234143 A1

906

ZAW’T ZIH’TIH
?un |ZAWS'TAWS af

dS
O
Patent Application Publication Nov. 25, 2004 Sheet 28 of 38 US 2004/0234143 A1

E
Patent Application Publication Nov. 25, 2004 Sheet 29 of 38 US 2004/0234143 A1

FIG. 29

C start D
S1
Determine B1 on RP1 co-located with block Bn
referred to by vector MV10 of BOO
S2
Regard MVO extending from B00 to B1 wn
as Scaling vector
S3
Perform scaling on vector MVO1 extending from
BO to RB01 on RP1 and on vector MVO2 extending
from BO to RBO2. On RP3
Patent Application Publication Nov. 25, 2004 Sheet 30 Of 38 US 2004/0234143 A1

099

T55ZG6

ZAW’T ZIY?'IH
•-,?796IG
ZAWS’TVNZIYJu’TH-I
\,096ZIH-
Patent Application Publication Nov. 25, 2004 Sheet 31 of 38 US 2004/0234143 A1

-5

ZIZAWNS’dIH-?'IHu

dS
O
?Ou04
Patent Application Publication Nov. 25, 2004 Sheet 32 of 38 US 2004/0234143 A1

ZS"SOI
Patent Application Publication Nov. 25, 2004 Sheet 33 of 38 US 2004/0234143 A1

ZAW’T ZIH’TIH
s^096
Patent Application Publication Nov. 25, 2004 Sheet 34 of 38 US 2004/0234143 A1
Patent Application Publication Nov. 25, 2004 Sheet 36 of 38 US 2004/0234143 A1

FIG. 36

eX208

eX2O3

eX204

eX205
eX115
Patent Application Publication Nov. 25, 2004 Sheet 37 of 38 US 2004/0234143 A1

p?uÐUuenO
X3
I
GI

'EIB
ZE
Patent Application Publication Nov. 25, 2004 Sheet 38 of 38 US 2004/0234143 A1

uO?npd}/7 sgnO?eblxd3e

six8S/
US 2004/0234143 A1 Nov. 25, 2004

IMAGE ENCODING METHOD AND PICTURE B3 is a block on which inter-picture prediction has been
DECODING METHOD performed with reference to a block RB31 and a block
RB32.
Technical Field
0017 AS described above, the current picture TP, which
0001. The present invention relates to a picture coding is a B picture, includes blocks which shall be inter-picture
method for coding a picture Signal and a picture decoding predicted with reference to respective blocks included in
method for decoding the coded picture Signal. other two pictures.
0.018 (2) Interpolative Prediction
BACKGROUND ART
0019 Referring to FIG. 2, a description is given of
0002 With the development of multimedia applications, interpolative prediction.
it has become common to handle information of all Sorts of
media Such as Video, audio and text in an integrated manner. 0020 FIG. 2 is a diagram explaining interpolative pre
diction.
0003) Against this backdrop, it becomes possible to 0021. A block RB1 and a block RB2 are two reference
handle media integrally by digitizing all types of media. blocks to be used for interpolative prediction, and a block
0004. However, since an enormous amount of data is PB indicates a predictive block obtained by interpolation.
contained in digitized pictures, a technique for compressing Here, a description is given, on the assumption that each
picture information is absolutely required to accumulate or block is sized 4x4 pixels.
transmit Such pictures. At the same time, it is also important 0022 X1(i) indicates a pixel value in the reference block
to Standardize compression techniques for the interoperabil RB1, X2(i) indicates a pixel value in the reference block
ity of compressed picture data. The Standard Specifications RB2, and P(i) indicates a pixel value in the predictive block
on picture compression techniques include H.261 and H.263 PB. The pixel value P(i) can be obtained using a linear
standards recommended by ITU-T (International Telecom prediction equation Such as below:
munication Union-Telecommunication Standardization Sec
tor) and MPEG (Moving Picture Experts Group)-1, MPEG
2, and MPEG-4 of ISO. Currently, ITU is in an effort of 0023. Here, A, B, and C are linear prediction coefficients.
Standardizing H.264 as the latest picture coding Standard, The use of linear prediction coefficients depends on the case;
which is called H.26L as a draft under standardization. only an average value (when A=1/2, B=1/2, and C=0) is used
0005 Inter-picture prediction utilizing motion compen as linear predictive coefficients in the case of MPEG-1 and
sation is a technique commonly used in moving picture -2 (more strictly, a value to be obtained by rounding, to the
coding methods such as MPEG-1, -2, and -4 and H.263. In closest integer value, the value obtained by dividing X1(i)+
motion compensation according to these moving picture X2(i) to half), whereas in another case, other values are
coding methods, a picture in an input picture Signal is explicitly set as linear prediction coefficients. When other
divided into rectangular areas (hereinafter referred to as values are Set as linear prediction coefficients, Such values
“blocks”) of a predetermined size, and predictive pixels are are carried from a picture coding apparatus to a picture
generated on a block-by-block basis from a motion vector decoding apparatus in a coded picture Signal.
indicating motion between pictures. 0024. Blocks on which inter-picture prediction is per
0006. In order to introduce inter-picture prediction using formed through pixel interpolation from plural reference
motion compensation, a description is given of the following pictures are referred to as "interpolation predictive blocks'.
items (1)-(6): A B picture is a picture capable of including interpolation
predictive blocks inside.
0007 (1) Concept of B picture;
0008 (2) Interpolative prediction; 0025. According to picture coding methods such as
MPEG-1 and MPEG-2, an interpolation predictive block
0009 (3) Picture number and reference index; included in a B picture means an interpolation predictive
block which is predicted from the reference picture RP1
0010 (4) Direct mode; located before the current picture TP in display order and the
0011 (5) Conventional picture coding apparatus; and reference picture RP3 located after the current picture TP in
0012 (6) Conventional picture decoding sprats. display order, as the block B1 shown in FIG. 1.
0026. In contrast, a recently proposed new B picture can
0013 (1) Concept of B Picture further include an interpolation predictive block that refers
0.014) Referring to FIG. 1, a description is given of B to, as reference pictures, two pictures both of which are
pictures (bi-predictive pictures) according to H.26L. located either before or after the current picture, as blockS
B2 and B3 shown in FIG. 1.
0.015 FIG. 1 is a conceptual diagram showing B pictures.
0016 A picture TP is the current B picture to be coded, 0027 FIG. 3 is a diagram explaining an example case
where two reference pictures of an interpolation predictive
whereas each of pictures RP1, RP2, RP3, and RP4 is an block are located, in display order, before the current picture
already coded picture. A block B1 is a block on which to be coded that includes the interpolation predictive block.
inter-picture prediction has been performed with reference
to a block RB1 and a block RB2. A block B2 is a block on 0028. A picture P7 is the current picture to be coded.
which inter-picture prediction has been performed with Each picture is coded in order of pictures P1, P2, P3, P4, P5,
reference to a block RB21 and a block RB22. And a block P6, and P7. This means that the pictures P1, P2, P5, and P6
US 2004/0234143 A1 Nov. 25, 2004

are coded before the current picture P7, and then stored into 0040. Meanwhile, reference indices are used to specify
the multi-frame buffer (memory). reference pictures to be used for inter-picture prediction of
0029. An interpolation predictive block included in the the current picture to be coded.
current picture P7 is coded with reference to pixels in the 0041 Reference indices are made up of first reference
picture P1 and the picture P6. indices for Specifying a first reference picture and Second
reference indices for Specifying a Second reference picture.
0030 FIG. 4 is a diagram explaining an example case
where two reference pictures of an interpolation predictive 0042 Referring to FIG. 5A, a description is given below
block are located, in display order, after the current picture of a method of assigning first reference indices and Second
to be coded that includes the interpolation predictive block. reference indices.

0.031) A picture P4 is the current picture to be coded. 0043 First, as first reference indices, values starting from
Each picture is coded in order of pictures P1, P2, P3, P4, and 0 are assigned, in order of proximity to the current picture,
P5. This means that the pictures P1, P2, and P3 are coded to reference pictures with earlier display times than the
before the current picture P4, and then stored into the current picture. After values Starting from 0 are assigned to
multi-frame buffer. all the reference pictures with earlier display times than the
current picture, the Subsequent values are assigned, in order
0032. An interpolation predictive block included in the of proximity to the current picture, to reference pictures with
current picture P4 is coded with reference to pixels in the later display times than the current picture.
picture P2 and the picture P3. 0044) Then, as second reference indices, values starting
0033. Note that a picture that can include blocks on from 0 are assigned, in order of proximity to the current
which inter-picture prediction shall be performed from a picture, to reference pictures with later display times than
Single reference picture is referred to as a Ppicture, whereas the current picture. After values Starting from 0 are assigned
a picture composed only of intra-picture predictive blockS to all the reference pictures with later display times than the
on which inter-picture prediction shall not be performed is current picture, the Subsequent values are assigned, in order
referred to as an I picture. of proximity to the current picture, to reference pictures with
earlier display times than the current picture.
0034. In H.26L, blocks in a B picture are coded using the 0045. For example, as shown in FIG. 5A, when 0 is
maximum of two reference pictures. Here, in order to indicated as a first reference index, the first reference picture
distinguish Such two reference pictures, one of the reference shall be a B picture with the picture number “ 14”, whereas
pictures is referred to as a first reference picture and the when 1 is indicated as a Second reference index, the Second
other a Second reference picture. Moreover, motion vectors reference picture shall be a B picture with the picture
to be determined based on the first reference picture and the number “13.
Second reference picture are respectively referred to as a first
motion vector and a Second motion vector. 0046. In H.26L, it is possible to arbitrarily change the
assignment of first reference indices and Second reference
0035). As for the current block B1 in FIG. 1, for example, indices to reference pictures by explicitly indicating Such
the picture RP1 is a first reference picture, the picture RP3 change by use of a buffer control signal (RPSL 7 within a
is a Second reference picture, the motion vector MV1 is a header 1 shown in FIG. 8) included in coded picture signal.
first motion vector, and the motion vector MV 2 is a second This change in the assignment of indices makes it possible
motion vector. Furthermore, prediction that is made based for an arbitrary reference picture inside the multi frame
only on a first reference picture is referred to as a first buffer to Serve as a reference picture with the Second
reference picture prediction, whereas prediction that is made reference index value of 0, and as shown in FIG. 5B, for
based only on a Second reference picture is referred to as example, different values are assigned as first reference
Second reference picture prediction. indices and Second reference indices for the respective
0.036 Note that there is no need to make a distinction picture numbers.
between first and Second reference pictures and motion 0047. Note that reference indices included in a block are
vectors for a block on which inter-picture prediction shall be represented as variable length code words, where a shorter
made using one reference picture. However, for explanation code is assigned to a Smaller value.
purposes, the reference picture and motion vector of a block 0.048 (4) Direct Mode
on which inter-picture prediction is performed using one
reference picture are referred to respectively as a first 0049 FIG. 6 is a diagram explaining direct mode
reference picture and a first motion vector. employed by a conventional picture coding apparatus.
0037 (3) Picture Number and Reference Index 0050 Here, direct mode is a mode for performing inter
picture prediction through pixel interpolation after determin
0038 FIGS.5A and 5B are diagrams explaining picture ing reference pictures and motion vectors of the current
numbers and reference indices. block to be coded, using a method described below, based on
0039) Picture numbers and reference indices are infor motion vectors used at the time of coding Such reference
mation used to uniquely identify reference pictures Stored in pictures and based on pictures which were referred to at the
the multi-frame buffer. In H.26L, a value which increments
time of coding Such reference pictures.
by one every time a picture is accumulated as a reference 0051 A picture TP is a B picture to be coded, whereas
picture in the memory is assigned to Such reference picture pictures RP1, RP2, RP3, and RP4 are decoded pictures
as its picture number. Stored in the multiframe buffer for use as reference pictures.
US 2004/0234143 A1 Nov. 25, 2004

Meanwhile, respective pictures indicated by dotted lines in picture number “15” in FIG. 5A, the picture RP4 in FIG. 6
FIG. 6 are non-reference pictures which shall not be stored corresponds to a B picture with the picture number “13” in
in the multi frame buffer. FIG. 5A, the picture RP1 in FIG. 6 corresponds to a B
0.052 Here, the first reference index RI1 of the picture picture with the picture number “ 14” in FIG. SA, and the
RP1 is 0, the first reference index RI1 of the picture RP2 is picture RP2 in FIG. 6 corresponds to a B picture with the
1, the second reference index RI2 of the picture RP3 is 0, and picture number “ 12” in FIG. 5A. Since non-reference pic
the second reference index RI2 of the picture RP4 is 1. tures represented by doted liens such as the picture TP, shall
not be referred to by other pictures, they shall not be stored
0053 A block B0 is a block to be coded in direct mode, in the multiframe buffer. Therefore, non-reference pictures
and a block B00 is a block in the picture RP3 which is are not be assigned reference indices used for referring to
co-located with the current block B0. Meanwhile, a block such pictures, as the pictures shown in FIG. 5A are.
RB01 is a reference block included in the picture RP1, and 0062) Note that in FIG. 6, the first motion vector MVO is
a block RB02 is a reference block included in the reference
picture RP3. A motion vector MV0, which is a first motion Schematically illustrated as a vector that goes beyond Space
vector to be used when coding the block B00, refers to the and time extending from the reference picture RP3 to the
picture RP1. Such motion vector MV0 shall be referred to as reference picture RP1. However, referring to FIG. 28, an
a Scaling Vector. actually stored first motion vector MVO indicates the amount
of a locational change from the block B00 to the block B3.
0.054 The following equations are used to determine a Therefore, the first motion vector MVO shown in FIG. 6 can
first motion vector MV01 and a second motion vector MV02 be determined from a vector extending from the block B00
which are used for predicting the current block B0 to be to the block B3 as well as from the temporal distance
coded: between RP1 and RP3.
0063. In this specification, Suppose that the motion vector
MV0 shown in FIG. 6 is actually stored for explanation
0055. In the above equations, the coefficients TR1, TR2 purposes.
and TR0 are values used to determine motion vectors for 0064 (5) Conventional Picture Coding Apparatus
direct mode, and an example of Such values is the difference
in display times between pictures. In FIG. 6, TR1=2, 0065 Referring now to FIG. 7, a description is given of
TR2=1, and TR0=3. a conventional picture coding apparatus.
0056 Assuming that an object that includes the current 0066 FIG. 7 is a block diagram showing the configura
block moves in a constant manner within the display, the tion of a conventional picture coding apparatus. The follow
first motion vector MV01 and the second motion vector ing gives a description of Such picture coding apparatus.
MV02 are determined by internally dividing the motion 0067. A picture coding apparatus 900 receives a picture
vector MV0 by (i) the difference in display times between Signal Img which is divided into blocks, and performs
the current picture TP and the first reference picture RP1 and processing on Such Signal on a block-by-block basis.
(ii) the difference in display times between the current
picture TP and the second reference picture RP2. 0068 A subtracter 901 subtracts each of predictive image
Signals Pre from the picture Signal Img, and outputs residual
0057. Note that not only the difference in display times Signals Res.
but also the difference in the picture numbers as well as the
number of pictures between pictures may also be used to 0069. An image coding unit 902 obtains the residual
determine the first motion vector MV01 and the second Signals Res So as to perform image coding processing Such
motion vector MVO2. as DCT conversion and quantization on each of Such
residual Signals Res, and outputs coded residual Signals ER
0.058 A set of the above coefficients TR1, TR2, and TR0 that include the quantized DCT coefficients and the like.
shall be referred to as a direct mode scaling coefficient SP.
0059. In MPEG-4, since there is only one following 0070 An image decoding unit 904 obtains the coded
reference picture at maximum in direct mode, a picture to be residual signals ER So as to perform image decoding pro
used as a Second reference picture is uniquely determined. In cessing Such as inverse quantization and inverse DCT con
H.26L, however, Since more than one picture can Serve as a version on each of Such signals, and outputs decoded
Second reference picture, the picture with the Second refer residual Signals DR.
ence index RI2 of 0 shall be used as the second reference 0.071) An adder 905 adds each of the decoded residual
picture in direct mode. Signals DR and each of the predictive image Signals Pre, and
outputs reconstructed image Signals Rec.
0060 Moreover, in direct mode, a direct mode scaling
coefficient SP transmitted for each picture is commonly used 0072 Reconstructed image signals Rec with the possi
by all the blocks included in each picture. Alternatively, bility to be referred to for inter-picture prediction of Subse
Scaling Shall be performed in proportion to the difference in quent pictures, will be stored into a multiframe buffer 907.
display times by use of display time information of each Since the multi frame buffer 907 has a limited amount of
picture. memory, picture data in the multi frame buffer 907 which
0061. When FIG. 6 is to correspond to FIG. 5A, the will not be used for later inter-picture prediction shall be
removed from the multi frame buffer 907.
picture TP in FIG. 6 corresponds to a B picture located in
the center (represented by the doted lines) in FIG. 5A, the 0073. A motion estimation unit 909 obtains a reference
picture RP3 in FIG. 6 corresponds to a B picture with the picture RP stored in the multi frame buffer 907 so as to
US 2004/0234143 A1 Nov. 25, 2004

perform motion estimation on it, Selects an optimum pre in addition to intra picture prediction, first reference picture
diction type on a block-by-block basis from among the prediction, Second reference picture prediction, and interpo
following prediction types: intra picture prediction, first lative prediction other than in direct mode.
reference picture prediction, Second reference picture pre 0081. Accordingly, when the selected prediction type
diction, and interpolative prediction using a predetermined
method (a prediction type to be selected depends on a picture indicates direct mode, a Switch 911 Switches to the “1” side,
and the reference indices rRI1 and rRI2 as well as the motion
type). Then, the motion estimation unit 909 outputs a first vectors SMV1 and SMV2 are used as the reference indices
motion vector MV1, a second motion vector MV2, a first RI1 and RI2 as well as the motion vectors MV1 and MV2.
reference indeX RI1, and a second reference index RI2 of the
current block to be coded. 0082) Meanwhile, when the prediction type indicates
0.074 An example method of selecting a prediction type other than direct mode, the Switch 911 Switches to the “0”
employed by the motion estimation unit 909 is to select a Side.
prediction type that generates the Smallest prediction error 0083. At the time of direct mode, the first motion vector
among all the prediction types. When intra picture prediction SMV1 used for coding a block in an already coded picture
is Selected as a prediction type, no motion vectors and is used as a Scaling vector. Moreover, a picture referred to by
reference indices are outputted. When first reference picture Such first motion vector SMV1 is used as one of the two
prediction is Selected, only a first reference indeX and a first reference pictures in direct mode. Therefore, among the
motion vector are outputted. When Second reference picture coded first reference indices RI1 and first motion vectors
prediction is Selected, only a Second reference indeX and a MV1, first reference indices RI1 and first motion vectors
Second motion vector are outputted. And when interpolative MV1 with the possibility to be used in direct mode for
prediction is Selected, a first reference index, a Second Subsequent pictures of the coded picture are Stored into the
reference index, a first motion vector, and a Second motion vector buffer 914.
vector are outputted.
0075 AS described above, in H.26L, a reference picture 0084. After a prediction type is determined, a first refer
with the second reference index rRI2 of 0 is selected as a ence index RI1 and a first motion vector MV1 are inputted
Second reference picture for direct mode. Accordingly, the to the multi frame buffer 907, and a reference block RB1
second reference index rRI2 with the value 0 is inputted to corresponding to Such inputted first reference indeX RI1 and
a vector buffer 914 and a direct mode processing unit 910. first motion vector MV1 is outputted from the multi frame
buffer 907 to a pixel interpolation unit 906. When a predic
0076 Stored in the vector buffer 914 are scaling vectors tion type requires two reference blocks, a reference block
rMV and the picture number of a picture referred to by each RB2 corresponding to a Second reference indeX RI2 and a
of Such Scaling vectors rMV. Since a reference picture second motion vector MV2 is outputted from the multi
including a block that has been coded using a Scaling vector frame buffer 907 to the pixel interpolation unit 906.
rMV is a reference picture indicated by a Second reference
index rRI2, the vector buffer 914 receives the Second ref 0085. When interpolative prediction is selected, the pixel
erence indeX rRI2 with the value 0, and outputs the Scaling interpolation unit 906 interpolates pixel values in positions
vector rMV and the first reference index rRI1 that indicates in the two reference blocks RB1 and RB2 corresponding to
a picture referred to by the scaling vector rMV. each other, and outputs an interpolated block ReP.
0077. The direct mode processing unit 910 receives a 0086. When the selected prediction type indicates inter
direct mode scaling coefficient SP, a scaling vector rMV, a polative prediction, a Switch 912 Switches to the “1” side,
first reference indeX rRI1, and a Second reference indeX and uses the interpolated block ReP as a predictive image
rRI2, and outputs a first motion vector SMV1, a second Signal Pre.
motion vector SMV2, a first reference index rRI1, and a
second reference index rRI2 for direct mode through the 0087 When first reference picture prediction is selected,
above-described direct mode processing. the multi frame buffer 907 outputs a reference block RB
corresponding to the first reference index RI1 and the first
0078) A prediction type selection unit 908 receives (i) the motion vector MV1. When second reference picture predic
picture Signal Img, (ii) the reference pictures RP, (iii) the tion is selected, the multi frame buffer 907 outputs a
reference indices rRI1 and rRI2 and the motion vectors
SMV1 and SMV2 which indicate the locations of reference reference block RB corresponding to the Second reference
index RI1 and the second motion vector MV1. Note that
blocks for “direct mode”, and (iv) the reference indices RI1 when intra picture prediction is Selected, the multi frame
and RI2 and the motion vectors MV1 and MV2 which
buffer 907 outputs a block RB that is made up of pixels
indicate the locations of reference blocks used for prediction generated as a result of intra picture prediction.
in “non-direct mode'. Then, the prediction type Selection
unit 908 determines whether to use direct mode or not for 0088. Meanwhile, when other than interpolative predic
block prediction, and outputs, to a variable length coding tion is indicated as a prediction type, the Switch Switches to
unit 903, type information PT indicating the determined the “0” side, and uses a reference block RB as a predictive
prediction type. image Signal Pre.
007.9 Here, the prediction type selection unit 908 selects 0089. The variable length coding unit 903 performs vari
a prediction type by Selecting, for example, a Smaller able length coding on the coded residual Signal ER, the
prediction error between prediction errors of input pixels at reference indices RI1 and RI2, the motion vectors MV1 and
the time of “direct mode” and “non-direct mode”.
MV2, the direct mode scaling coefficient SP, and the type
0080 Thus, direct mode is further included as a predic information PT, and incorporates the resultant into a bit
tion type to be selected by the motion estimation unit 909, stream BSO so as to output it.
US 2004/0234143 A1 Nov. 25, 2004

0090 FIG. 8 is a conceptual diagram showing the format information of each picture), and the type information PT.
of the bit stream BSO. The image decoding unit 952 receives the coded residual
Signals ER, performs inverse quantization and inverse DCT
0091 FIG. 8 shows the format of a part in the bit stream conversion on each of Such residual Signals ER, and outputs
BS0 where information equivalent to one picture is decoded residual signals DR. The adder 953 adds each of the
included.
decoded residual signals DR and each of the predictive
0092 Such part includes the header 1, a coded first block image Signals Pre, and outputs decoded picture Signals DIm
Signal 2 of a block which was coded in direct mode, and a to outside the picture decoding apparatus 950. The multi
coded second block signal 3 of a block which was coded frame buffer 958 stores some of the decoded picture signals
using interpolative prediction other than in direct mode. DIm necessary for inter-picture prediction.
0093. The coded second block signal 3 includes: a coded 0101 The vector buffer 960 holds a scaling vector rMV
type signal 9 obtained by coding type information PT, a and information used for identifying a picture refereed to by
coded first index Signal 10 and a coded Second indeX signal such scaling vector rMV (the first reference index rRI1).
11 obtained by coding reference indices RI1 and RI2; and a Moreover, the vector buffer 960 receives the second refer
coded MV1 signal 12 and a coded MV2 signal 13 obtained ence indeX rRI2 with the value 0, and outputs the Scaling
by coding motion vectors MV1 and MV2. In the coded vector rMV and the first reference index rRI1.
second block signal 3, the coded first index signal 10 and the 0102) The direct mode processing unit 954 performs the
coded second index signal 11 as well as the coded MV1 Same processing as that of the direct mode processing unit
signal 12 and coded MV2 signal 13 are included in the bit 910 Shown in FIG. 7.
Stream BSO in the order shown in FIG. 8.
0094) Determination on whether to use a reference index 0103) When the prediction type is other than direct mode,
RI1 or RI2 is made according to the coded type signal 9, and the Switch 955 Switches to the “0” side. Then, the multi
a first reference picture and a Second reference picture are frame buffer 958 obtains the reference indices RI1, and RI2
as well as the motion vectors MV1 and MV2.
determined by the data positions in the bit stream BS0 of the
coded first indeX signal 10 and the coded Second indeX signal 0104. When the prediction type indicates direct mode, the
11. Switch 955 Switches to the “1” side. Then, the multi frame
0.095 Meanwhile, the coded first block signal 2, which buffer 958 obtains the reference indices rRI1 and rRI2, as
well as the motion vectors SMV1 and SMV2 as the reference
indicates that a block has been coded in direct mode, indices RI1 and RI2 as well as the motion vectors MV1 and
includes the coded type Signal 8 obtained by coding the type MV2.
information PT, but does not include information indicating
reference indices and motion vectors. 0105. When the selected prediction type indicates inter
0096) The header 1 includes: first time difference infor polative prediction, the multi frame buffer 958 outputs a
mation 4, Second time difference information 5, and time reference block RB1 corresponding to the first reference
difference information 6 obtained by coding the coefficients index RI1 and the first motion vector MV1 as well as a
TR1, TR2, and TR0 shown in FIG. 6; and RPSL 7 that reference block RB2 corresponding to the Second reference
indicates a change in the assignment of first reference index RI2 and the second motion vector MV2. Subse
indices or Second reference indices to reference pictures. quently, the pixel interpolation unit 957 interpolates pixel
Note, however, that when display time information of each values corresponding to the respective two reference blockS
picture is used for Scaling a motion vector in direct mode, RB1 and RB2, and outputs an interpolated block ReP.
Such information is not described in the header, and only 0106 When first reference picture prediction is selected,
display time information is described in the header. the multi frame buffer 958 outputs a reference block RB
0097 (6) Conventional Picture Decoding Apparatus corresponding to the first reference index RI1 and the first
motion vector MV1. When second reference picture predic
0.098 FIG. 9 is a block diagram showing the configura tion is selected, the multi frame buffer 958 outputs a
tion of a conventional picture decoding apparatus. reference block RB corresponding to the Second reference
0099. Here, a multi frame buffer 958, a pixel interpola index RI2 and the second motion vector MV2. Note that
tion unit 957, a vector buffer 960, and a direct mode when intra picture prediction is Selected, the multi frame
processing unit 954 equipped to a picture decoding appara buffer 958 outputs a block RB that is made up of pixels
tus 950 respectively have the same functions as those of the generated as a result of intra picture prediction.
multiframe buffer 907, the pixel interpolation unit 906, the 0107 Meanwhile, when interpolative prediction is indi
vector buffer 914, and the direct mode processing unit 910 cated as a prediction type, the Switch956 Switches to the “0”
of the picture coding apparatus 900 shown in FIG. 7. A side, and the interpolated block ReP is used as a predictive
detailed description of them, therefore, is omitted. More image Signal Pre.
over, the same signals in FIG. 9 as those shown in FIG. 7
are assigned the Same numbers, and no description is given 0.108 Meanwhile, when other than interpolative predic
of them. tion is indicated as a prediction type, the Switch956 Switches
to the “1” side, and a reference block RB is used as a
0100. The variable length decoding unit 951 receives the predictive image Signal Pre.
bit stream BSO so as to perform variable length decoding on
it, and outputs the coded residual signals ER, the motion 0109 Amongst the decoded first reference indices RI1
vectors MV1 and MV2, the reference indices RI1 and RI2, and first motion vectors MV1, first reference indices RI1 and
the direct mode scaling coefficients SP (or the display time first motion vectors MV1 with the possibility to be used in
US 2004/0234143 A1 Nov. 25, 2004

direct mode for Subsequent pictures of a decoded picture are frame buffers 907 and 958 as well as information used to
Stored into the vector buffer 960. identify pictures referred to by such first motion vectors.
0110. The picture decoding apparatus 950 with the above 0118. Since the amount of memory required to store such
configuration decodes the bit stream BS0 through the above first motion vectors and picture identification information is
described processing, and outputs the resultant as decoded proportional to the number of reference pictures, the vector
picture Signals DIm. buffers 914 and 960 that hold such information are required
to have vast Storage capacity. In other words, Since an
0111 Here, in the above-described conventional picture enormous amount of information is to be Stored in the Vector
coding apparatus 900 and picture decoding apparatus 950, a buffers 914 and 960, load is inevitably placed on the
B picture is coded with reference to (i) two forward pictures, inputting/outputting of Such information. As a result, coding
(ii) two backward pictures, or (iii) one forward picture and processing becomes complicated, leading to a decrease in
one backward picture. Regarding general Video, however, a coding efficiency.
higher coding efficiency can be achieved by referring to one
forward picture and one backward picture rather than refer 0119) It is an object of the preset invention to provide a
ring to two forward pictures or two backward pictures, Since picture coding method and a picture decoding method
highly accurate interpolative prediction is possible in the capable of offering an improved coding efficiency.
former case.
Disclosure of Invention
0112 However, in the above-described conventional pic
ture coding apparatus 900 and picture decoding apparatus 0120 In order to achieve the above object, the picture
950, since the value 0 of a second reference index rRI2 is coding method according to the present invention is a
assigned to an arbitrary reference picture in the multi frame picture coding method for coding a current picture to be
buffers 907 and 908 as described with reference to FIG. 5B. coded on a block-by-block basis with reference to already
Therefore, in direct mode, a B picture is coded with refer coded pictures, comprising: a Second reference picture
ence to two forward pictures or two backward pictures in Selection Step of Selecting, as a Second reference picture, any
many cases, which causes a problem that coding efficiency one of coded pictures located after the current picture in
in coding of B pictures is lowered. display order; a first reference picture Specification Step of
Specifying, as a first reference picture, any one of coded
0113 Also, as shown in FIG. 6, in direct mode, the pictures based on a motion vector used for coding a prede
current picture TP is coded by use of the first reference termined block in the Second reference picture; a predictive
picture RP1 and the second reference picture RP3 as well as image generation Step of Specifying a block in the first
the scaling vectors MVO (the first motion vectors MV0). reference picture and a block in the Second reference picture
Here, the Second reference picture is a picture whose Second based on the motion vector, and generating a predictive
reference index RI2 value is 0, and the first reference picture image from Said two blocks through pixel interpolation; a
is a picture which is referred to by the first motion vector coding Step of coding a difference between the predictive
MV0 of the block B00 in the second reference picture. image and an image of a current block to be coded; and an
output Step of outputting Specification information for Speci
0114. Therefore, in order to realize direct mode, the fying the Second reference picture, together with a result of
picture coding apparatus 900 is required to hold the first the coding performed in the coding Step.
motion vectors MVO used for coding the second reference
picture RP3 as well as information used to identify the 0121 Here, in the output step, the specification informa
picture RP1 referred to by each of such motion vectors tion that includes a Second reference indeX indicating the
MVO. Second reference picture may be outputted. Moreover, in the
output Step, information indicating a method for changing
0115 By the same token, the picture decoding apparatus Second reference indices may be outputted as the Specifica
950 is also required to hold the first motion vectors MV0 of tion information, Said Second reference indices being
a decoded picture as well as information used to identify the assigned to the respective coded pictures in order to identify
picture RP1 referred to by each of such motion vectors Said coded pictures.
MVO.
0122) Accordingly, when remapping occurs, Since a
0116. In H.263, a motion vector can be switched to coded picture located after the current picture in display
another one on a 4x4 block basis at the minimum and a order is Selected as the Second reference picture, it becomes
reference picture can be Switched to another one on an 8x8 possible to prevent two coded pictures located before the
block basis at the minimum. Therefore, in the case of a current picture in display order from being refereed to as in
QCIF-sized image (176x144 blocks), the memory is the conventional cases, and therefore to improve the prob
required to be capable of holding information for identifying ability that two coded pictures located before and after the
1584 motion vectors and 396 reference pictures per refer current picture in display order are refereed to. This conse
ence picture. quently improves coding efficiency. Furthermore, Since the
Specification information is outputted, it is possible to cor
0117 Moreover, in H.26L, since a change can be freely rectly decode a result of the coding executed in the coding
made in the assignment of reference indices to reference Step, by Selecting the same picture as the Second reference
pictures, any reference picture in the multi frame buffer can picture Selected at the time of coding, based on Such
be a reference picture corresponding to the Second reference Specification information.
index RI2=0. This requires the picture coding apparatus 900
and the picture decoding apparatus 950 to hold the first 0123. Also, the picture coding method according to the
motion vectors of all reference pictures Stored in the multi present invention is a picture coding method for coding a
US 2004/0234143 A1 Nov. 25, 2004

current picture to be coded on a block-by-block basis with dictive image from Said two blocks through pixel interpo
reference to already coded pictures, comprising: a Second lation; and a coding Step of coding a difference between the
reference picture Selection Step of Selecting, as a Second predictive image and an image of a current block to be
reference picture, a coded picture from among coded pic coded.
tures that are located after the current picture in display 0127 Here, in the re-assignment step, it may be prohib
order, Said coded picture being assigned a Smallest Second ited to change the Second reference index of the coded
reference index of all Second reference indices assigned to picture which is assigned the Second reference indeX indi
the respective coded pictures in order to identify Said coded cating 0 in the assignment Step.
pictures, a first reference picture specification Step of Speci
fying, as a first reference picture, any one of coded pictures 0128. Accordingly, in the assignment step and the re
based on a motion vector used for coding a predetermined assignment Step, Since the Second reference indeX indicating
block in the Second reference picture, a predictive image 0 is assigned to a coded picture that is located after the
generation Step of Specifying a block in the first reference current picture to be coded in display order, a coded picture
picture and a block in the Second reference picture based on that is located after the current picture to be coded in display
the motion vector, and generating a predictive image from order is Selected as the Second reference picture. As a result,
Said two blocks through pixel interpolation; and a coding when remapping occurs, Since a coded picture located after
Step of coding a difference between the predictive image and the current picture in display order is Selected as the Second
an image of a current block to be coded. reference picture, it becomes possible to prevent two coded
0.124. Here, in the Second reference picture Selection step, pictures located before the current picture in display order
the coded picture with the Smallest Second reference indeX from being refereed to as in the conventional cases, and
may be specified according to assignment information indi therefore to improve the probability that two coded pictures
cating how the Second reference indices are assigned, and located before and after the current picture in display order
Said Specified coded picture may be Selected as the Second are refereed to. This consequently improves coding effi
reference picture. ciency.
0.125. Accordingly, when remapping occurs, Since a 0129. Furthermore, the picture coding method according
coded picture located after the current picture in display to the present invention is a picture coding method for
coding a current picture to be coded on a block-by-block
order is Selected as the Second reference picture, it becomes basis with reference to already coded pictures, comprising:
possible to prevent two coded pictures located before the an assignment Step of assigning, to respective coded pic
current picture in display order from being refereed to as in tures, first reference indices for identifying Said coded
the conventional cases, and therefore to improve the prob pictures, a re-assignment Step of re-assigning the first ref
ability that two coded pictures located before and after the erence indices to predetermined coded pictures or slices in
current picture in display order are refereed to. This conse a manner different from the manner used in the assignment
quently improves coding efficiency. Furthermore, Since the Step; a first reference picture Selection Step of Selecting, as
picture with the Smallest reference indeX is Selected as the a first reference picture, a coded picture assigned a first
Second reference picture, it is possible to correctly decode a reference indeX indicating a predetermined value, from
result of the coding performed in the coding Step, by among coded pictures located before the current picture in
uniquely Selecting the Same picture as the Second reference display order; a predictive image generation Step of gener
picture Selected at the time of coding. ating a predictive image through pixel interpolation based on
0.126 Here, the picture coding method according to the a predetermined block in the first reference picture, and a
present invention is a picture coding method for coding a coding Step of coding a difference between the predictive
current picture to be coded on a block-by-block basis with image and an image of a current block to be coded, wherein
reference to already coded pictures, comprising: an assign in the re-assignment Step, the first reference indeX indicating
ment Step of assigning, to respective coded pictures, Second Said predetermined value is assigned preferentially to the
reference indices for identifying Said coded pictures in a coded picture which is referred to for a long period of time.
manner in which a Second reference indeX indicating 0 is 0.130. Accordingly, when a scene change occurs, Since
assigned to any one of coded pictures located after the remapping is performed in a manner in which the first
current picture in display order; a re-assignment Step, in reference indeX indicating O is preferentially assigned to a
which it is possible to re-assign the Second reference indices coded picture in a memory that Stores coded pictures to be
in a manner different from the manner used in the assign referred to for a long period of time, and the coded picture
ment Step, of assigning the Second reference indeX indicat with Such first reference index of 0 is selected as the first
ing 0 only to any one of the coded pictures located after the reference picture, it is possible to improve the probability
current picture in display order, when re-assigning the that a coded picture Similar to the current picture to be coded
Second reference indices, a Second reference picture Selec is referred to. This consequently improves coding efficiency.
tion Step of Selecting, as a Second reference picture, the
coded picture assigned the Second reference indeX indicating 0131 Moreover, the picture coding method according to
0 from among the coded pictures located after the current the present invention is a picture coding method for coding
picture in display order; a first reference picture Specification a current picture to be coded on a block-by-block basis with
Step of Specifying, as a first reference picture, any one of the reference to already coded pictures, comprising: an assign
coded pictures based on a motion vector used for coding a ment Step of assigning, to respective coded pictures, Second
predetermined block in the Second reference picture; a reference indices for identifying Said coded pictures, a
predictive image generation Step of Specifying a block in the Second reference picture Selection Step of Selecting, as a
first reference picture and a block in the Second reference Second reference picture, a coded picture which is assigned
picture based on the motion vector, and generating a pre a Second reference indeX indicating 0, from among coded
US 2004/0234143 A1 Nov. 25, 2004

pictures located after the current picture in display order; a motion vector of the current block which refers to the second
first reference picture Specification Step of Specifying, as a reference picture is calculated, both based on Said Second
first reference picture, any one of the coded pictures based motion vector, and (iii) the predictive image is generated
on a motion vector used for coding a predetermined block in through pixel interpolation from a block in the first reference
the Second reference picture; a predictive image generation picture referred to by the third motion vector and a block in
Step of Specifying a block in the first reference picture and the second reference picture referred to by the fourth motion
a block in the Second reference picture based on the motion VectOr.
vector, and generating a predictive image from Said two 0.136 Accordingly, since the same first reference picture
blocks through pixel interpolation; and a coding Step of is commonly used for each block in the current picture to be
coding a difference between the predictive image and an coded regardless of which picture a first motion vector of the
image of a current block to be coded, wherein in the Second reference picture refers to, it becomes unnecessary to
assignment Step, the Second reference indeX indicating 0 is Store information for identifying the first reference picture
prohibited from being assigned to a coded picture that has into the memory, as in the conventional cases. This can
been located before the current picture in display order. reduce the amount of memory as well as Simplifying coding
0132) Accordingly, Since the Second reference index indi processing So as to improve coding efficiency.
cating 0 is prohibited from being assigned to a coded picture 0.137 Moreover, the picture decoding method according
that has been located before the current picture to be coded to the present invention is a picture decoding method for
in display order, Such coded picture that has been located decoding a coded picture on a block-by-block basis with
after the current picture in display order shall never be reference to already decoded pictures, comprising: an
Selected as the Second reference picture. As a result, it obtainment Step of obtaining Specification information and a
becomes possible to omit the processing for unnecessarily coded signal, the Specification information being used for
Storing motion vectors used for coding the blocks on Such Specifying, as a Second reference picture, a decoded picture
coded pictures, which consequently leads to improved cod to be referred to for decoding a current picture to be decoded
ing efficiency. and the coded Signal being related to Said current picture; a
0.133 Here, the picture coding method with the above Second reference picture Selection Step of Selecting the
configuration may further comprise a deletion Step of delet Second reference picture from among decoded pictures that
ing, from a memory Storing motion vectors used for coding are located after the current picture in display order based on
the coded pictures, a motion Vector used for coding the the Specification information; a first reference picture Speci
coded picture that has been located before the current picture fication Step of Specifying, as a first reference picture, any
in display order. one of decoded pictures based on a motion vector used for
decoding a predetermined block in the Second reference
0134. Accordingly, it becomes possible to reduce the picture; a predictive image generation Step of Specifying a
amount of memory by preventing motion vectors unneces block in the first reference picture and a block in the Second
Sary for coding from being Stored into the memory. reference picture based on the motion vector, and generating
a predictive image from Said two blocks through pixel
0135 Also, the picture coding method according to the interpolation; and a decoding Step of decoding a current
present invention is a picture coding method for coding a block to be decoded based on the predictive image and the
current picture to be coded on a block-by-block basis with coded Signal.
reference to already coded pictures, comprising: a Selection
Step of Selecting, from among a plurality of coded pictures 0.138 Here, in the obtainment step, the specification
Stored in a storage unit, a coded picture corresponding to a information that includes a Second reference indeX indicat
first reference indeX as a first reference picture and a coded ing the Second reference picture may be obtained, and in the
picture corresponding to a Second reference indeX as a Second reference picture Selection Step, the Second reference
Second reference picture, by use of Said first reference indeX picture indicated by Said Second reference indeX may be
and Said Second reference indeX assigned to Said plurality of Selected. Furthermore, in the obtainment Step, the Specifi
coded pictures, a predictive image generation Step of gen cation information that indicates a method for changing
erating a predictive image from a block in the first reference Second reference indices may be obtained, said Second
picture and a block in the Second reference picture through reference indices being assigned to the respective decoded
pixel interpolation; and an output Step of coding a prediction pictures in order to identify Said decoded pictures, and in the
error which is a difference between the current picture and Second reference picture Selection Step, the decoded picture
the predictive image, and outputting a coded picture Signal indicated by Said method for changing the Second reference
that includes a Signal obtained by coding Said prediction indices may be selected as the Second reference picture.
error, wherein in the Selection Step, the coded picture which 0.139. Accordingly, it becomes possible to correctly
Satisfies the following condition is Selected as the first decode a picture that has been decoded using the picture
reference picture from among the coded pictures: a coded coding method according to the present invention, by
picture which is located before the current picture in display uniquely Selecting the Second reference picture based on the
order and which is assigned a Smallest reference index, and Specification information.
in the predictive image generation step, (i) a second motion
vector of a block which refers to the first reference picture 0140. In addition to the above, the picture coding method
is calculated, based on a first motion vector used to code Said according to the present invention may be a picture coding
block which is in the Second reference picture and co method, comprising: a first Step of Selecting either at least a
located with a current block to be coded in the current first or Second reference frame from among a plurality of
picture, (ii) a third motion vector of the current block which coded frames Stored in a storage unit by use of a first relative
refers to the first reference picture is calculated and a fourth indeX and a Second relative indeX assigned to Said coded
US 2004/0234143 A1 Nov. 25, 2004

frames, in order to Select a first reference frame and a Second method, comprising: a first Step of Selecting either at least a
reference frame to be referred to when obtaining a block on first or Second reference frame from among a plurality of
a current frame to be coded by means of motion compen coded frames Stored in a storage unit by use of a first relative
sation; a Second step of generating, through pixel interpo indeX and a Second relative indeX assigned to Said coded
lation, a predictive image from a block on at least either the frames, in order to Select a first reference frame and a Second
first or Second reference frame obtained by means of motion reference frame to be referred to when obtaining a block on
compensation; and a third step of coding a prediction error a current frame to be coded by means of motion compen
which is a difference between the input current frame and the sation; a Second step of generating, through pixel interpo
predictive image, and outputting a coded picture signal that lation, a predictive image from a block on at least either the
includes a Signal obtained by coding Said prediction error, first or Second reference frame obtained by means of motion
wherein in the first Step, a reference frame which Satisfies the compensation; and a third Step of coding a prediction error
following condition is Selected as the Second reference which is a difference between the input current frame and the
frame from among the coded frames: a reference frame predictive image, and outputting a coded picture signal that
which is located after the current frame in display order and includes a Signal obtained by coding Said prediction error,
which is assigned a Smallest Second relative index, and in the
Second step, (i) a frame is regarded as the first reference wherein in the first step, a frame which is located after the
frame, Said frame being refereed to by a motion vector used current frame in display order is Selected as the Second
for motion-compensating a block which is in the Second reference frame, from among the coded frames, in the
reference frame and co-located with a predetermined block Second step, (i) a frame is regarded as the first reference
in the current frame, (ii) based on said motion vector, a frame, Said frame being refereed to by a motion vector used
motion vector to the first reference frame and a motion for motion-compensating a block which is in the Second
vector to the Second reference frame are calculated using a reference frame and co-located with a predetermined block
predetermined method, and (iii) the predictive image is in the current frame, (ii) based on said motion vector, a
generated through pixel interpolation from a block obtained motion vector to the first reference frame and a motion
from the motion vector to the first reference frame and a vector to the Second reference frame are calculated using a
block obtained from the motion vector to the second refer predetermined method, and (iii) the predictive image is
ence frame. generated through pixel interpolation from a block obtained
from the motion vector to the first reference frame and a
0.141. In addition to the above, the picture decoding block obtained from the motion vector to the second refer
method according to the present invention may be a picture
decoding method, comprising: a first Step of obtaining a ence frame, and in the third Step, the Second relative indeX
coded picture Signal that includes a Signal obtained by indicating the Second reference frame Selected in the first
coding a prediction error, a Second step of Selecting either at Step is incorporated into the coded picture Signal.
least a first or Second reference frame from among a plurality 0143. In addition to the above, the picture decoding
of decoded frames Stored in a Storage unit by use of a first method according to the present invention may be a picture
relative indeX and a Second relative indeX assigned to Said decoding method, comprising: a first Step of obtaining a
decoded frames, in order to Select a first reference frame and coded picture Signal that includes a Signal obtained by
a Second reference frame to be referred to when obtaining a coding a prediction error, a Second step of Selecting either at
block on a current frame to be decoded by means of motion least a first or Second reference frame from among a plurality
compensation; a third Step of generating, through pixel of decoded frames Stored in a Storage unit by use of a first
interpolation, a predictive image from a block on at least relative indeX and a Second relative indeX assigned to Said
either the first or Second reference frame obtained by means decoded frames, in order to Select a first reference frame and
of motion compensation; a fourth Step of generating a a Second reference frame to be referred to when obtaining a
decoded picture of a frame from the predictive image and the block on a current frame to be decoded by means of motion
decoded prediction error; and a fifth Step of Storing, in the compensation; third step of generating, through pixel inter
Storage unit, a decoded picture of a frame with a possibility polation, a predictive image from a block on at least either
to be used for inter-frame prediction, wherein in the Second the first or Second reference frame obtained by means of
Step, a reference frame which Satisfies the following condi motion compensation; a fourth Step of generating a decoded
tion is Selected as the Second reference frame from among picture of a frame from the predictive image and the decoded
the decoded frames: a reference frame which is located after prediction error; and a fifth Step of Storing, in the Storage
the current frame in display order and which is assigned a unit, a decoded picture of a frame with a possibility to be
Smallest Second relative index, and in the third step, (i) a used for inter-frame prediction, wherein in the first Step, the
frame is regarded as the first reference frame, Said frame Second relative indeX is obtained from the coded picture
being refereed to by a motion vector used for motion Signal including Second relative indices, in the Second Step,
compensating a block which is in the Second reference frame a frame indicated by the Second relative indeX obtained in
and co-located with a predetermined block in the current the first Step is regarded as the Second reference frame, in the
frame, (ii) based on said motion vector, a motion vector to third step, (i) a frame is regarded as the first reference frame,
the first reference frame and a motion vector to the Second Said frame being refereed to by a motion vector used for
reference frame are calculated using a predetermined motion-compensating a block which is in the Second refer
method, and (iii) the predictive image is generated through ence frame and co-located with a predetermined block in the
pixel interpolation from a block obtained from the motion current frame, (ii) based on Said motion vector, a motion
vector to the first reference frame and a block obtained from vector to the first reference frame and a motion vector to the
the motion vector to the Second reference frame. Second reference frame are calculated using a predetermined
0142. In addition to the above, the picture coding method method, and (iii) the predictive image is generated through
according to the present invention may be a picture coding pixel interpolation from a block obtained from the motion
US 2004/0234143 A1 Nov. 25, 2004

vector to the first reference frame and a block obtained from with a predetermined block in the current frame and from a
the motion vector to the Second reference frame. temporal interval between the first reference frame and the
0144. In addition to the above, the picture coding method Second reference frame, a Second motion vector extending
according to the present invention may be a picture coding from the first block to a block which is in the first reference
method, comprising: a first Step of Selecting either at least a frame and co-located with a block referred to by the first
first or Second reference frame from among a plurality of motion vector is calculated, (ii) based on said calculated
coded frames Stored in a storage unit by use of a first relative Second motion vector, a motion vector to the first reference
indeX and a Second relative indeX assigned to Said coded frame and a motion vector to the Second reference frame are
frames, in order to Select a first reference frame and a Second calculated using a predetermined method, and (iii) the
reference frame to be referred to when obtaining a block on predictive image is generated through pixel interpolation
a current frame to be coded by means of motion compen from a block obtained from the motion vector to the first
sation; a Second step of generating, through pixel interpo reference frame and a block obtained from the motion vector
lation, a predictive image from a block on at least either the to the Second reference frame.
first or Second reference frame obtained by means of motion 0146 In addition to the above, the picture coding method
compensation; and a third step of coding a prediction error according to the present invention may be a picture coding
which is a difference between the input current frame and the method, comprising: a first Step of Selecting either at least a
predictive image, and outputting a coded picture signal that first or Second reference frame from among a plurality of
includes a Signal obtained by coding Said prediction error, coded frames Stored in a storage unit by use of a first relative
wherein in the first Step, a reference frame which Satisfies the indeX and a Second relative indeX assigned to Said coded
following condition is Selected as the first reference frame frames, in order to Select a first reference frame and a Second
from among the coded frames: a reference frame which is reference frame to be referred to when obtaining a block on
located before the current frame in display order and which a current frame to be coded by means of motion compen
is assigned a Smallest first relative index, and in the Second sation; a Second step of generating, through pixel interpo
Step, (i) from a first motion vector used for motion-com lation, a predictive image from a block on at least either the
pensating a first block which is in the Second reference frame first or Second reference frame obtained by means of motion
and co-located with a predetermined block in the current compensation; and a third Step of coding a prediction error
frame and from an interval between the first reference frame which is a difference between the input current frame and the
and the Second reference frame, a Second motion vector predictive image, and outputting a coded picture signal that
extending from the first block to a block which is in the first includes a Signal obtained by coding Said prediction error,
reference frame and co-located with a block referred to by wherein in the first Step, the first reference frame is Selected
the first motion vector is calculated, (ii) based on said from frames that are located before the current frame in
calculated Second motion vector, a motion vector to the first display order, in the Second step, (i) from a first motion
reference frame and a motion vector to the Second reference vector used for motion-compensating a first block which is
frame are calculated using a predetermined method, and (iii) in the Second reference frame and co-located with a prede
the predictive image is generated through pixel interpolation termined block in the current frame and from a temporal
from a block obtained from the motion vector to the first interval between the first reference frame selected in the first
reference frame and a block obtained from the motion vector Step and the Second reference frame, a Second motion vector
to the Second reference frame. extending from the first block to a block which is in the first
0145. In addition to the above, the picture decoding reference frame and co-located with a block referred to by
method according to the present invention may be a picture the first motion vector is calculated, (ii) based on Said
decoding method, comprising: a first Step of obtaining a calculated Second motion vector, a motion vector to the first
coded picture Signal that includes a Signal obtained by reference frame Selected in the first Step and a motion vector
coding a prediction error, a Second step of Selecting either at to the Second reference frame are calculated using a prede
least a first or Second reference frame from among a plurality termined method, and (iii) the predictive image is generated
of decoded frames Stored in a Storage unit by use of a first through pixel interpolation from a block obtained from the
relative indeX and a Second relative indeX assigned to Said motion vector to the first reference frame selected in the first
decoded frames, in order to Select a first reference frame and step and a block obtained from the motion vector to the
a Second reference frame to be referred to when obtaining a Second reference frame, and in the third step, the first
block on a current frame to be decoded by means of motion relative indeX indicating the first reference frame Selected in
compensation; a third Step of generating, through pixel the first Step is incorporated into the coded picture Signal.
interpolation, a predictive image from a block on at least 0147 In addition to the above, the picture decoding
either the first or Second reference frame obtained by means method according to the present invention may be a picture
of motion compensation; a fourth Step of generating a decoding method, comprising: a first Step of obtaining a
decoded picture of a frame from the predictive image and the coded picture Signal that includes a Signal obtained by
decoded prediction error; and a fifth Step of Storing, in the coding a prediction error, a Second step of Selecting either at
Storage unit, a decoded picture of a frame with a possibility least a first or Second reference frame from among a plurality
to be used for inter-frame prediction, wherein in the Second of decoded frames Stored in a Storage unit by use of a first
Step, a reference frame which Satisfies the following condi relative indeX and a Second relative indeX assigned to Said
tion is Selected as the first reference frame from among the decoded frames, in order to Select a first reference frame and
decoded frames: a reference frame which is located before a Second reference frame to be referred to when obtaining a
the current frame in display order and which is assigned a block on a current frame to be decoded by means of motion
Smallest first relative index; and in the third step, (i) from a compensation; a third Step of generating, through pixel
first motion vector used for motion-compensating a first interpolation, a predictive image from a block on at least
block which is in the Second reference frame and co-located either the first or Second reference frame obtained by means
US 2004/0234143 A1 Nov. 25, 2004

of motion compensation; a fourth Step of generating a decoded residual signals, an addition unit operable to add
decoded picture of a frame from the predictive image and the each of the decoded residual signals and each of predictive
decoded prediction error; and a fifth Step of Storing, in the image Signals, and output decoded pictures, a storage unit
Storage unit, a decoded picture of a frame with a possibility operable to Store Said decoded pictures, a direct mode
to be used for inter-frame prediction, wherein in the first Second reference frame Selection unit operable to Select, as
step, the first relative index is obtained from the coded a Second reference frame, a reference frame that is assigned
picture Signal including first relative indices, and in the a Smallest Second relative index; a direct mode vector/
Second step, (i) from a first motion vector used for motion relative indeX generation unit operable to generate motion
compensating a first block which is in the Second reference vectors to a first reference frame and to the Second reference
frame and co-located with a predetermined block in the frame using a predetermined method, based on a motion
current frame and from a temporal interval between the first vector referred to by a block that is in the second reference
reference frame Selected in the first Step and the Second frame and co-located with a current block to be coded, Said
reference frame, a Second motion vector extending from the first reference frame being a frame referred to by the motion
first block to a block which is in the first reference frame and vector referred to by said block collocated with the current
co-located with a block referred to by the first motion vector block, and a pixel interpolation unit operable to perform
is calculated, (ii) based on Said calculated Second motion pixel interpolation on the first and Second reference frames
vector, a motion vector to the first reference frame Selected as well as on two reference blocks referred to by the motion
in the first Step and a motion vector to the Second reference vectors corresponding to the respective reference frames,
frame are calculated using a predetermined method, and (iii) and output the predictive image Signal.
the predictive image is generated through pixel interpolation
from a block obtained from the motion vector to the first 0150. Furthermore, the recording medium according to
reference frame obtained in the first Step and a block the present invention is a recording medium in which a
obtained from the motion vector to the second reference program is Stored by a computer, the program being intended
frame. for performing coding or decoding processing on a moving
picture described in any one of the aforementioned picture
0.148. In addition to the above, the picture coding appa coding and picture decoding methods as well as causing the
ratus according to the present invention may be a picture computer to execute the coding or decoding processing on a
coding apparatus, comprising: a difference calculation unit moving picture described in any one of the aforementioned
operable to obtain a picture Signal, calculate a difference picture coding and picture decoding methods.
between Said picture Signal and a predictive image, and
output Said difference as a residual signal; an image coding 0151. Note that it is possible to embody the present
unit operable to perform image coding on Said differential invention as the following that use the above-described
Signal, and output Said coded signal as a coded residual picture coding method or picture decoding method: a picture
Signal; an image decoding unit operable to decode Said coding apparatus, a picture decoding apparatus, a program,
coded residual Signal, and output Said decoded Signal as a and a recording medium Storing Such program.
decoded residual signal; an addition unit operable to add the
decoded residual Signal and the predictive image, and output BRIEF DESCRIPTION OF DRAWINGS
a reconstructed image, a direct mode Second reference frame 0152 FIG. 1 is a conceptual diagram showing B pictures.
Selection unit operable to Select, as a Second reference
frame, a reference frame that Satisfies the following condi 0153 FIG. 2 is a diagram explaining interpolative pre
tion from among coded frames Stored in a storage unit: a diction.
reference frame that is located after a current frame to be
coded and that is assigned a Smallest Second relative index; 0154 FIG. 3 is a diagram explaining an example case
a direct mode Vector/relative indeX generation unit operable where two reference pictures of an interpolation predictive
to generate motion vectors to a first reference frame and to block are located before a current picture to be coded in
the Second reference frame using a predetermined method, display order that includes the interpolation predictive
based on a motion vector used for motion-compensating a block.
block that is in the Second reference frame and co-located O155 FIG. 4 is a diagram explaining an example case
with a predetermined block in the current frame, Said first where two reference pictures of an interpolation predictive
reference frame being a frame referred to by the motion block are located after a current picture to be coded in
vector used for Said motion compensation; a pixel interpo display order that includes the interpolation predictive
lation unit operable to perform pixel interpolation on two block.
reference blocks referred to by the motion vector to the first
reference frame and the motion vector to the Second refer 0156 FIGS.5A and 5B are diagrams explaining picture
ence frame, and output the result as the predictive image; numbers and reference indices.
and a variable length coding unit operable to perform O157 FIG. 6 is a diagram explaining direct mode
variable length coding on a prediction error, and output a employed by a conventional picture coding apparatus.
coded Signal.
0149. In addition to the above, the picture decoding 0158 FIG. 7 is a block diagram showing a configuration
apparatus according to the present invention may be a of a conventional picture coding apparatus.
picture decoding apparatus, comprising: a variable length 0159 FIG. 8 is a conceptual diagram showing a format
decoding unit operable to obtain a coded picture Signal, of a bit stream.
perform variable length decoding on Said Signal, and output
coded residual signals, an image decoding unit operable to 0160 FIG. 9 is a block diagram showing a configuration
decode each of Said coded residual signals, and output of a conventional picture decoding apparatus.
US 2004/0234143 A1 Nov. 25, 2004

0.161 FIG. 10 is a block diagram showing a picture 0182 FIG. 31 is a block diagram showing a picture
coding apparatus according to a first embodiment of the coding apparatus according to a ninth embodiment of the
present invention. present invention.
0162 FIG. 11 is a diagram showing a partial structure of 0183 FIG. 32 shows a format of a bit stream BS3
a bit Stream according to the first embodiment. according to the ninth embodiment.
0163 FIG. 12 is a flowchart showing an operation of 0.184 FIG. 33 is a block diagram showing a picture
coding a picture in direct mode in the first embodiment. decoding apparatus according to a tenth embodiment of the
0164 FIGS. 13A and 13B are diagrams explaining an present invention.
operation to be performed when remapping takes place in 0185 FIGS. 34A, 34B, and 34C are diagrams explaining
the first embodiment. a recording medium according to an eleventh embodiment
0.165 FIG. 14 is a diagram explaining an example dis of the present invention.
play order of pictures in the first embodiment. 0186 FIG. 35 is a block diagram showing an overall
0166 FIG. 15 is a block diagram showing a picture configuration of a content Supply System that realizes a
decoding apparatus according to a Second embodiment of content distribution Service according to a twelfth embodi
the present invention. ment of the present invention.
0167 FIG. 16 is a flowchart showing an operation of 0187 FIG. 36 is a diagram showing a cellular phone
decoding a picture in direct mode in the Second embodiment. according to the twelfth embodiment.
0168 FIG. 17 is a block diagram showing a picture 0188 FIG. 37 is a block diagram showing the cellular
coding apparatus according to a third embodiment of the phone according to the twelfth embodiment.
present invention. 0189 FIG. 38 is a block diagram showing an overall
0169 FIG. 18 is a diagram showing a structure of a configuration of a digital broadcasting System according to
header of a bit stream in the third embodiment. the twelfth embodiment

0170 FIG. 19 is a flowchart showing an operation of BEST MODE FOR CARRYING OUT THE
coding a picture in direct mode in the third embodiment. INVENTION
0171 FIG. 20 is a block diagram showing a picture 0.190) (First Embodiment)
decoding apparatus according to a fourth embodiment of the
present invention. 0191 The following describes the picture coding appa
ratus according to the first embodiment of the present
0172 FIG. 21 is a flowchart showing an operation of invention with reference to the figures.
decoding a picture in direct mode in the fourth embodiment.
0173 FIG. 22 is a block diagram showing a picture 0.192 FIG. 10 is a block diagram showing the picture
coding apparatus according to a fifth embodiment of the coding apparatus according to the first embodiment.
present invention. 0193 Such picture coding apparatus 100, which obtains
0.174 FIG. 23 is a diagram explaining remapping per a picture Signal Img with the contents representing pictures
formed by a management unit in the fifth embodiment. and codes Such picture Signal Img on a block-by-block basis,
is comprised of a Subtracter 101, an image decoding unit
0175 FIG. 24 is a flowchart showing an operation per 104, a variable length coding unit 103, a motion estimation
formed in the fifth embodiment. unit 109, a prediction type selection unit 108, Switches 111
and 112, a pixel interpolation unit 106, a direct mode
0176 FIG. 25 is a diagram showing a configuration of a processing unit 110, a vector buffer 114, a multiframe buffer
picture coding apparatus according to a variation of the fifth 107, and a picture selection unit 113.
embodiment.
0194 The subtracter 101 subtracts predictive image sig
0177 FIG. 26 is a block diagram showing a picture nals Pre from the picture Signal Img, and outputs the
decoding apparatus according to a sixth embodiment of the resultant as residual Signals Res.
present invention.
0.195 The image coding unit 102 obtains the residual
0.178 FIG. 27 is a block diagram showing a picture Signals Res from the Subtracter 101 So as to perform image
coding apparatus according to a Seventh embodiment of the coding processing Such as DCT conversion and quantization
present invention. on each of Such residual signals Res, and outputs coded
0179 FIG. 28 is a conceptual diagram showing direct residual signals ER that include quantized DCT coefficients
and the like.
mode according to the Seventh embodiment.
0180 FIG. 29 is a flowchart showing a process of 0196. The image decoding unit 104 obtains the coded
calculating motion vectors in the Seventh embodiment. residual Signals ER from the image coding unit 102 So as to
perform image decoding processing Such as inverse quan
0181 FIG. 30 is a block diagram showing a picture tization and inverse DCT conversion on each of Such coded
decoding apparatus according to an eighth embodiment of residual Signals ER, and outputs decoded residual signals
the present invention. DR.
US 2004/0234143 A1 Nov. 25, 2004

0197) The adder 105 adds each of the decoded residual Second reference indeX rR2, and outputs the first reference
Signals DR and each of the predictive image Signals Pre, and indeX RI1 indicating a picture referred to by Such Scaling
outputs the resultant as reconstructed image Signals Rec. vector rMV as a first reference index rRI1.
0198 Upon the receipt of the reconstructed image signals 0204 After obtaining a direct mode scaling coefficient
Rec from the adder 105, the multi frame buffer 107 stores SP, a scaling vector rMV, a first reference index rRI1, and a
signals with the possibility to be referred to for inter-picture Second reference indeX rRI2, the direct mode processing unit
prediction of Subsequent pictures, out of Such received 110 performs processing based on the above-described
reconstructed image Signals Rec. Since the multi frame direct mode on the current block to be coded, and outputs the
buffer 107 has a limited amount of memory, the multiframe resultant as a first motion vector SMV1, a second motion
buffer 107 removes picture data which will not be used for vector SMV2, a first reference index rRI1, and a second
later inter-picture prediction, out of the reconstructed image
signals Rec obtained from the adder 105. reference index rRI2. Here, instead of obtaining the direct
mode Scaling coefficient SP, the direct mode processing unit
0199 The motion estimation unit 109 obtains a picture 110 may obtain display time information that indicates the
stored in the multiframe buffer 907 as a reference picture RP display time of each of the reference pictures specified by
So as to perform motion estimation on it, Selects an optimum the first reference indices rRI1 and display time information
prediction type for Such picture using a predetermined that indicates the display time of each of the reference
method (a prediction type to be selected depends on a picture pictures Specified by the Second reference indiceS rRI2 as
type) from among the following prediction types: intra well as display time information indicating the display time
picture prediction, first reference picture prediction, Second of the current picture to be coded. In this case, the direct
reference picture prediction, and interpolative prediction. mode processing unit 110 shall calculate the difference in
Here, the motion estimation unit 109 selects, for example, a display times between each picture from the display time of
prediction type as the optimum prediction type that gener each picture indicated by Such display time information, and
ates the Smallest prediction error among all the prediction performs Scaling on the Scaling vector rMV using the ratio
types. of the display time difference between each of the pictures.
0200 When interpolative prediction is selected, the Accordingly, the direct mode processing unit 110 can deter
motion estimation unit 109 outputs the first motion vector mine a first motion vector SMV1 and a second motion vector
MV1, the second motion vector MV2, the first reference SMVS as in the case of using a direct mode Scaling coeffi
index RI1, and the second reference index RI2 of the current cient SP.
block to be coded. When intra picture prediction is selected, 0205 The prediction type selection unit 108 obtains (i)
the motion estimation unit 109 outputs no motion vectors the picture signal Img, (ii) the reference pictures RP, and (iii)
and reference indices as described above. When first refer
the first reference indices rRI1, the second reference indices
ence picture prediction is Selected, the motion estimation rRI2, the first motion vectors SMV1, and the second motion
unit 109 outputs only the first motion vector MV1 and the vectors SMV2 outputted from the direct mode processing
first reference index RI1. When second reference picture unit 110, as well as obtaining the first reference indices RI1,
prediction is selected, the motion estimation unit 109 out the second reference indices RI2, the first motion vectors
puts only the second motion vector MV2 and the second MV1, and the second motion vectors MV2 outputted from
reference index RI2.
the motion estimation unit 109. Then, the prediction type
0201 The vector buffer 114 obtains, from the motion selection unit 108 selects a prediction type for the current
estimation unit 109, the first motion vector MV1, the second block to be coded based on Such obtained information. In
motion vector MV2, the first reference index RI1, and the other words, the prediction type selection unit 108 judges
second reference index RI2 on a block-by-block basis. Then, whether to apply direct mode or not to Such current block to
on the basis of Such obtainment, the vector buffer 114 stores be coded. Furthermore, the prediction type Selection unit
scaling vectors rMV used for direct mode as well as the 108 outputs, to the variable length coding unit 108, type
picture numbers and reference indices indicating pictures information PT indicating the Selected prediction type.
referred to by the respective scaling vectors rMV. 0206 Here, in order to select a selection type, the pre
0202) In other words, in direct mode, since a first motion diction type Selection unit 108 compares, for example,
vector used to code a block in a coded picture is used as a prediction errors at the time of applying/not applying direct
scaling vector rMV and a picture referred to by such first mode to the current block. When judging that a prediction
motion vector Serves as its first reference picture, the vector error is Smaller when direct mode is applied, the prediction
buffer 114 stores first motion vectors MV1 and first refer type selection unit 108 selects a prediction type that utilizes
ence indices RI1 with the possibility to be used for direct direct mode. When judging, on the other hand, that a
mode, out of the first motion vectors MV1, the second prediction error is Smaller when direct mode is not applied,
motion vectors MV2, the first reference indices RI1, and the the prediction type selection unit 108 selects a prediction
second reference indices RI2 outputted from the motion type that does not utilize direct mode.
estimation unit 109.
0207 Furthermore, when selecting to apply direct mode
0203 Furthermore, the vector buffer 114 obtains a second as a prediction type, the prediction type Selection unit 108
reference index rRI2 from the picture selection unit 113. causes the Switch 111 to change the contact from 0 to 1.
After obtaining the second reference index rRI2 from the Accordingly, a first reference indeX rRI1, a Second reference
picture selection unit 113, the vector buffer 114 outputs, as index rRI2, a first motion vector SMV1, and a second motion
a scaling vector rMV, the first motion vector MV1 of a vector SMV2 outputted from the direct mode processing unit
predetermined block included in a picture indicated by Such 110 are transmitted to the multi frame buffer 107 via the
US 2004/0234143 A1 Nov. 25, 2004

Switch 111 as a first reference index RI1, a second reference 0214. The variable length coding unit 103 performs vari
index RI2, a first motion vector MV1, and a second motion able length coding on the coded residual signals ER, the first
vector MV2. reference indices RI1, the second reference indices RI2, the
0208 Meanwhile, when selecting not to apply direct first motion vectors MV1, the second motion vectors MV2,
mode as a prediction type, the prediction type Selection unit the direct mode Scaling coefficients SP, and the type infor
108 causes the Switch 111 to change the contact from 1 to 0. mation PT, and outputs the resultant as a bit stream BS. In
Accordingly, a first reference indeX RI1, a Second reference the case where display time information is used by the direct
index RI2, a first motion vector MV1, and a second motion mode processing unit 110 instead of the direct mode Scaling
vector MV2 outputted from the motion estimation unit 109 coefficients SP, the variable length coding unit 103 performs
are transmitted to the variable length coding unit 103 and the variable length coding on the display time information used
multi frame buffer 107 via the Switch 111. for the above-performed Scaling without variable length
0209 Furthermore, when selecting a prediction type coding the direct mode Scaling coefficients SP, and incor
other than interpolative prediction, the prediction type Selec porates the variable length-coded display time information
tion unit 108 causes the Switch 112 to change the contact into the bit stream BS.
from 1 to 0, whereas the prediction type selection unit 108 0215 FIG. 11 is a diagram showing a partial structure of
causes the Switch 112 to change the contact from 0 to 1 when the bit stream BS.
Selecting interpolative prediction as a prediction type. As a
result, predictive image Signals Pre obtained by the Sub 0216) The bit stream BS is structured on a picture-by
tracter 101 and the adder 105 are assorted into ones output picture basis, where information about the header and blockS
ted from the multiframe buffer 107 and ones from the pixel is included on a picture-by-picture basis.
interpolation unit 106, depending on the prediction type of
each predictive image Signal Pre. 0217. The part of the bit stream BS shown in FIG. 11,
0210. At the time of interpolative prediction, the multi which is the equivalent of a picture that includes blockS
frame buffer 107 obtains a first reference index RI1 (rRI1) coded in direct mode, is made up of a header 21, a coded first
and a first motion vector MV1 (SMV1) based on the outputs block signal 22, and a coded Second block Signal 23.
from the motion estimation unit 109 and the direct mode
processing unit 110, and outputs, to the pixel interpolation 0218. The coded first block signal 22, which indicates a
unit 106, a reference block RB1 corresponding to Such first block coded in direct mode, includes a coded type Signal 28
reference index RI1 and first motion vector MV1. Further obtained by coding type information PT that indicates the
more, the multi frame buffer 107 obtains a second reference use of direct mode.
index RI2 (rRI2) and a second motion vector MV2 (SMV2) 0219. The coded second block signal 23, which indicates
based on the outputs from the motion estimation unit 109 a block coded in interpolative prediction other than in direct
and the direct mode processing unit 110, and outputs, to the mode, includes: a coded type signal 29 obtained by coding
pixel interpolation unit 106, a reference block RB2 corre type information PT that indicates the non-use of direct
sponding to Such Second reference indeX RI2 and Second mode; a coded first index signal 30 obtained by coding a first
motion vector MV2.
reference indeX RI1; a coded Second index Signal 31
0211. At the time of first reference picture prediction, the obtained by coding a Second reference indeX RI2; a coded
multi frame buffer 107 outputs a reference block RB corre MV1 signal 32 obtained by coding a first motion vector
sponding to the first reference index RI1 and the first motion MV1; and a coded MV2 signal 33 obtained by coding a
vector MV1. Meanwhile, at the time of second reference Second motion vector MV2.
picture prediction, the multi frame buffer 107 outputs a
reference block RB corresponding to the Second reference 0220. The header 21 includes: coded remapping infor
index RI2 and the second motion vector MV2. Note that mation 27 obtained by coding remapping information indi
when intra picture prediction is Selected, the multi frame cating a change in the assignment of reference indices to the
buffer 107 outputs a block RB that is made up of pixels picture; a coefficient TR1 included in a direct mode Scaling
generated as a result of intra picture prediction. coefficient SP shown in FIG. 6, i.e. first time difference
0212. After obtaining the reference blocks RB1 and RB2 information 24 obtained by coding the difference in display
from the multiframe buffer 107, the pixel interpolation unit times between the current picture to be coded and the first
106 interpolates the pixel values of co-located pixels of the reference picture; a coefficient TR2 included in the direct
respective reference blocks RB1 and RB2, and outputs the mode Scaling coefficient SP, i.e. Second time difference
resultant as an interpolated block ReP. information 25 obtained by coding the difference in display
times between the current picture to be coded and the Second
0213 The contacts of the Switch 112 are changed accord reference picture; and a coefficient TR0 included in the
ing to a prediction type indicated by the prediction type direct mode scaling coefficient SP, i.e. time difference infor
Selection unit 108 as described above. Therefore, when mation 26 obtained by coding the difference in display times
prediction other than interpolative prediction is Selected as a between the first reference picture and the Second reference
prediction type, a block RB from the multiframe buffer 107 picture. Note that when the direct mode processing unit 110
is transmitted to the Subtracter 101 and the adder 105 via the uses not the direct mode scaling coefficient SP but display
Switch 112 as a predictive image Signal Pre. When interpo time information, the header includes a Signal obtained by
lative prediction is Selected as a prediction type, on the other coding the display time information of the picture, instead of
hand, an interpolated block ReP from the pixel interpolation the first time difference information 24, the second time
unit 106 is transmitted to the Subtracter 101 and the adder difference information 25 and the time difference informa
105 via the Switch 112 as a predictive image signal Pre. tion 26.
US 2004/0234143 A1 Nov. 25, 2004

0221) Here, the picture selection unit 113 in the present with the picture number 14, 1 is assigned as a first reference
embodiment Selects, as a Second reference picture, a picture indeX to a picture with the picture number 12, and 2 is
to be used for direct mode from among pictures Stored in the assigned as a first reference indeX to a picture with the
multi frame buffer 107. picture number 10.
0222 Stated another way, the picture selection unit 113 0233. At the completion of assigning first reference indi
Selects, as the Second reference picture for direct mode, a ces to all the pictures preceding the current picture in display
picture with the Smallest Second reference indeX among the order, then, 3, 4, and 5 are assigned as first reference indices
pictures located after the current picture to be coded in to all the pictures located after the current picture in display
display order. order, in order of proximity to Such current picture to be
0223) Then, the picture selection unit 113 outputs, to the coded. More specifically, 3 is assigned as a first reference
vector buffer 114 and the direct mode processing unit 110, indeX to a picture with the picture number 15, 4 is assigned
the Second reference indeX rRI2 of Such Selected Second as a first reference indeX to a picture with the picture number
reference picture. Note that the picture selection unit 113 can 13, and 5 is assigned as a first reference indeX to a picture
know about the display order of each picture by using with the picture number 11.
information about display order attached to each picture. 0234 Subsequently, 0, 1, and 2 are assigned as second
0224 FIG. 12 is a flowchart showing the operation of reference indices to all the pictures located after the current
coding a picture in direct mode in the present embodiment. picture in display order, in order of proximity to Such current
picture to be coded. More Specifically, 0 is assigned as a
0225. First, the picture coding apparatus 100 causes the Second reference indeX to a picture with the picture number
picture Selection unit 113 to Select a picture with the Smallest 15, 1 is assigned as a Second reference indeX to a picture
Second reference indeX as the Second reference picture, from with the picture number 13, and 2 is assigned as a Second
among the Subsequent pictures of the current picture to be reference indeX to a picture with the picture number 11.
coded in display order (Step S100).
0235. At the completion of assigning second reference
0226) Next, the picture coding apparatus 100 causes the indices to all the pictures located after the current picture in
vector buffer 114 to specify, as a scaling vector rMV, the first display order, then, 3, 4, and 5 are assigned as Second
motion vector MV1 of a block that is co-located with the reference indices to all the pictures preceding the current
current block and included in the Second reference picture picture in display order, in order of proximity to Such current
Selected in the above manner, and to specify, as a first picture to be coded. More specifically, 3 is assigned as a
reference picture, a picture referred to by Such first motion Second reference indeX to a picture with the picture number
vector MV1 (Step S102). 14, 4 is assigned as a Second reference indeX to a picture
0227. Then, the picture coding apparatus 100 generates a with the picture number 12, and 5 is assigned as a Second
predictive image in direct mode through the direct mode reference index to a picture with the picture number 10.
processing unit 110, the multiframe buffer 107, and the pixel 0236 FIG. 13B shows first reference indices and second
interpolation unit 106, utilizing the Second reference picture reference indices assigned to the respective pictures as well
Selected in the above manner as well as the first reference
picture and the scaling vector rMV specified in the above as remapping information in the case where remapping was
carried out.
manner (Step S104).
0228. After generating the predictive image, the picture 0237 When remapping takes place, the first reference
coding apparatus 100 generates a prediction error from the indeX assigned to a picture with the picture number "14 is
current block to be coded and Such predictive image (Step changed, for example, from “0” to “2', the first reference
S106), and codes such generated prediction error (Step indeX assigned to a picture with the picture number "12' is
S108). changed from “1” to “0”, and the first reference index
assigned to a picture with the picture number “10” is
0229 Here, a description is given of the operation to be changed from “2” to “1”. Moreover, the second reference
performed in the first embodiment when a change is made in indeX assigned to a picture with the picture number "14 is
first reference indices and Second reference indices to be changed from “3’ to “0”, the second reference index
assigned to other pictures with respect to the current picture assigned to a picture with the picture number “15” is
to be coded (Such operation is hereinafter refereed to as changed from “0” to “2”, and the second reference index
“remapping”). assigned to a picture with the picture number “11” is
0230 FIGS. 13A and 13B are diagrams explaining the changed from “2” to “3”.
operation to be carried out in the first embodiment when 0238 Remapping information is made up of numerical
remapping takes place. values indicating changes in reference indices of respective
0231 FIG. 13A shows first reference indices and second pictures Subject to Such changes. Remapping information
shown in FIG. 13B is for the second reference indices. For
reference indices assigned to the respective pictures before example, when the picture which is assigned the Second
remapping takes place (in the default State). reference index of “0” is changed from a picture with the
0232 Before remapping takes place, i.e. in the default picture number “15” to a picture with the picture number
State, 0, 1, and 2 are assigned as first reference indices to all “14, the value “-1” obtained by subtracting the picture
the pictures preceding the current picture in display order number “15” from the picture number “ 14” shall be included
which is indicated by dotted lines in FIG. 13A, in order of in the remapping information in association with the picture
proximity to Such current picture to be coded. More spe number “ 14''. Moreover, when the picture which is assigned
cifically, 0 is assigned as a first reference indeX to a picture the Second reference index of "2 is changed from a picture
US 2004/0234143 A1 Nov. 25, 2004

with the picture number “11” to a picture with the picture 0246 When the pictures P3 and P4 are coded, the refer
number “15”, the value "+4' obtained by subtracting the ence picture P2 Serves as a reference picture which is located
picture number “11” from the picture number “15” shall be later in display order than Such current pictures to be coded.
included in the remapping information in association with Meanwhile, when the pictures P6 and P7 are coded, the
the picture number “15”. Similarly, when the picture which reference picture P2 Serves as a reference picture which is
is assigned the Second reference index of '3' is changed located earlier in display order than Such current pictures to
from a picture with the picture number "14" to a picture with be coded.
the picture number “11”, the value “-3’ obtained by Sub 0247 When pictures have such a relationship as shown in
tracting the picture number “ 14” from the picture number
“11” shall be included in the remapping information in FIG. 14, the reference picture P2 cannot serve as a reference
association with the picture number “11”. Note that when picture located after a current picture to be coded in display
remapping as shown in FIG. 13B is carried out, the picture order when coding the pictures Subsequent to Such reference
which is assigned the second reference index of “1” shall not picture P2. In other words, once a reference picture comes
be changed, but the value “0” is included in the remapping before a current picture to be coded in display order, Such
information in association with the picture number “13”. reference picture Shall never come after a current picture to
be coded in display order again.
0239). Such remapping information is managed by the
management unit in the picture coding apparatus 100 not 0248 Stated another way, at the point of time when a
illustrated in the figures, and is used by the motion estima picture stored in the multi frame buffer 107 comes before a
tion unit 109, the prediction type selection unit 108, the current picture in display order, it becomes possible to delete
multiframe buffer 107, and the picture selection unit 113 to Such reference picture's first motion vectors and information
Specify the first reference indices and Second reference used to identify a picture referred to by each of such first
indices. motion vectors stored in the vector buffer 114.

0240. In the present embodiment with the above configu 0249. Therefore, when pictures have such a relationship
ration, the picture Selection unit 113 Selects, from among the as shown in FIG. 14, the vector buffer 114 is required to hold
pictures located after the current picture in display order, a only the first motion vectors of the reference pictures located
picture with the Smallest Second reference indeX as the after the current picture in display order as well as infor
Second reference picture. Therefore, when no remapping mation used to identify pictures referred to by Such first
takes place, a picture with the picture number “15” and with motion vectors. Accordingly, when pictures have a relation
the Second reference indeX “0” is used as a Second reference ship as shown in FIG. 14, it becomes possible to reduce the
picture in direct mode, whereas when remapping does take amount of memory required for the vector buffer 114.
place, a picture with the picture number “13' and with the 0250) Note that in the present embodiment, a picture with
Second reference indeX “1” is used as a Second reference the Smallest Second reference indeX is Selected as a Second
picture in direct mode. reference picture in direct mode, from among pictures that
0241 Accordingly, even when remapping takes place, are located after the current picture to be coded in display
Since a picture located after the current picture is used as a order. However, a picture to be Selected as a Second refer
Second reference picture at the time of coding in direct ence picture does not have to be a picture with the Smallest
mode, it is not possible for forward two pictures or backward reference index, as long as Such picture to be Selected
two pictures to be used as first reference pictures or Second Satisfies a condition under which a Second reference picture
reference pictures. This allows a forward picture and a in direct mode can be uniquely determined from among
backward picture to be used respectively as a first reference reference pictures located after the current picture in display
picture and a Second reference picture without fail, which order. In Such case too, there arises no possibility to refer to
consequently leads to an improved coding efficiency. only pictures preceding the current picture to be coded,
which results in an increased coding efficiency. Furthermore,
0242 Here, a description is given of the memory capacity a Second reference picture in direct mode may also be a
of the picture coding apparatus 100 according to the present picture which is closest to the current picture to be coded in
embodiment. display order among the reference pictures Subsequent to
Such current picture in display order. This can also improve
0243 FIG. 14 is a diagram explaining an example dis coding efficiency as in the above case.
play order of pictures.
0244 AS FIG. 14 shows, pictures P1-P7 are displayed in 0251 Furthermore, it is also possible that a second ref
erence picture for direct mode is a picture with the Smallest
the order of the picture P1, the picture P3, the picture P4, the Second reference index among the pictures that are located
picture P2, the picture P6, the picture P7, and the picture P5, after the current picture to be coded in display order and
and are coded in the order of the picture P1, the picture P2, further that a first reference picture for direct mode is a
the picture P3, the picture P4, the picture P5, the picture P6, picture with the Smallest first reference indeX among the
and the picture P7 to be stored into the multi frame buffer pictures that are located before the current picture to be
107. coded in display order. Coding efficiency can be improved in
0245 Here, B pictures shall not be used as reference this case too, as in the above case.
pictures. Therefore, the multi frame buffer 107 stores the 0252) (Second Embodiment)
pictures P1, P2, and P5 which are either I picture or Ppicture
to be handled as reference pictures, and deletes the pictures 0253) The following describes the picture decoding appa
P3, P4, P6, and P7 which are B pictures not to be handled ratus according to the Second embodiment of the present
as reference pictures. invention with reference to the figures.
US 2004/0234143 A1 Nov. 25, 2004

0254 FIG. 15 is a block diagram showing the picture 0262. In other words, in direct mode, since a first motion
decoding apparatus according to the Second embodiment. vector MV1 used to decode a block in a decoded picture is
0255 Such picture decoding apparatus 150, which used as a Scaling vector and a picture referred to by Such first
decodes the bit stream BS outputted from the picture coding motion vector MV1 serves as its first reference picture, the
vector buffer 160 stores first motion vectors MV1 and first
apparatus 100 in the first embodiment, is comprised of a reference indices RI1 with the possibility to be used for
variable length decoding unit 151, an image decoding unit direct mode, out of the first motion vectors MV1, the second
152, an adder 153, Switches 155 and 156, a direct mode motion vectors MV2, the first reference indices RI1, and the
processing unit 154, a pixel interpolation unit 157, a multi second reference indices RI2 outputted from the variable
frame buffer 158, a vector buffer 160, and a picture selection length decoding unit 151.
unit 159.
0263. Furthermore, the vector buffer 160 obtains a sec
0256 The variable length decoding unit 151 obtains the ond reference index rRI2 from the picture selection unit 159.
bit Stream BS from outside the picture decoding apparatus After obtaining the second reference index rRI2 from the
150, performs variable length decoding on such bit stream picture selection unit 159, the vector buffer 160 outputs, as
BS, and outputs the coded residual Signals ER, the motion a scaling vector rMV, the first motion vector MV1 of a
vectors MV1 and MV2, the reference indices RI1 and RI2, predetermined block included in a picture indicated by Such
the direct mode Scaling coefficients SP, and the type infor Second reference indeX rR2, and outputs the first reference
mation PT. Here, when the bit stream BS does not include indeX RI1 indicating a picture referred to by Such Scaling
coded direct mode scaling coefficients SP but coded display vector rMV as a first reference index rRI1.
time information, the variable length decoding unit 151 0264. After obtaining a direct mode scaling coefficient
outputs the display time information instead of the direct SP, a scaling vector rMV, and a first reference index rRI1, the
mode scaling coefficients SP. direct mode processing unit 154 performs processing based
0257) Furthermore, the variable length decoding unit 151 on the above-described direct mode on the current block to
changes the contacts of the Switches 155 and 156 according be decoded, and outputs, as the resultant, a first motion
to a prediction type indicated by each type information PT. vector SMV1, a second motion vector SMV2, a first refer
In other words, when a prediction type is direct mode, the ence index rRI1, and a second reference index rRI2. Here,
variable length decoding unit 151 changes the contact of the when obtaining display time information of each picture
Switch 155 from “0” to “1”, whereas when a prediction type instead of the direct mode scaling coefficient SP, the direct
is other than direct mode, the variable length decoding unit mode processing unit 154 shall calculate the difference in
151 changes the contact of the switch 155 from “1” to “0”. display times between each picture from the display time of
Meanwhile, when a prediction type is interpolative predic each picture indicated by Such display time information,
tion, the variable length decoding unit 151 changes the performs Scaling on the Scaling vector rMV using the ratio
contact of the Switch 156 from “1” to “0”, whereas when a of the display time difference between each of the pictures.
prediction type is other than interpolative prediction, the Accordingly, the direct mode processing unit 154 can deter
variable length decoding unit 151 changes the contact of the mine a first motion vector SMV1 and a second motion vector
Switch 156 from “O'” to “1”. SMV2 as in the case of using a direct mode Scaling coeffi
cient SP.
0258. The image decoding unit 152 obtains the coded 0265 At the time of interpolative prediction, the multi
residual signals ER from the variable length decoding unit frame buffer 158 obtains a first reference index RI1 (rRI1)
151 So as to perform image decoding processing Such as and a first motion vector MV1 (SMV1) based on the outputs
inverse quantization and inverse DCT conversion on each of from the variable length decoding unit 151 and the direct
Such coded residual signals ER, and outputs decoded mode processing unit 154, and outputs, to the pixel inter
residual signals DR. polation unit 157, a reference block RB1 corresponding to
0259. The adder 153 adds each of the decoded residual Such first reference index RI1 and first motion vector MV1.
Signals DR and each of the predictive image Signals Pre, and Furthermore, the multi frame buffer 158 obtains a second
outputs the resultant as decoded image Signals DIm to reference index RI2 (rRI2) and a second motion vector MV2
outside the picture decoding apparatus 150. (SMV2) based on the outputs from the variable length
decoding unit 151 and the direct mode processing unit 154,
0260. Upon the receipt of the decoded image signals and outputs, to the pixel interpolation unit 157, a reference
DIm, the multi frame buffer 158 stores signals with the block RB2 corresponding to Such Second reference index
possibility to be referred to for inter-picture prediction of RI2 and second motion vector MV2.
Subsequent pictures, out of the decoded image Signals DIm. 0266. At the time of first reference picture prediction, the
Since the multi frame buffer 158 has a limited amount of
memory, the multi frame buffer 158 removes picture data multi frame buffer 158 outputs a reference block RB corre
which will not be used for later inter-picture prediction, out sponding to the first reference index RI1 and the first motion
vector MV1. Meanwhile, at the time of second reference
of the above-obtained decoded image Signals DIm. picture prediction, the multi frame buffer 158 outputs a
0261) The vector buffer 160 obtains, from the motion reference block RB corresponding to the Second reference
estimation unit 109, the first motion vectors MV1, the index RI2 and the second motion vector MV2. Note that
Second motion vectors MV2, the first reference indices RI1, when intra picture prediction is Selected, the multi frame
and the Second reference indices RI2. Then, the vector buffer buffer 158 outputs a block RB that is made up of pixels
160 stores scaling vectors used for direct mode as well as the generated as a result of intra picture prediction.
picture numbers and reference indices indicating pictures 0267. After obtaining the reference blocks RB1 and RB2
referred to by the respective Scaling vectors. from the multiframe buffer 158, the pixel interpolation unit
US 2004/0234143 A1 Nov. 25, 2004

157 generates interpolated pixels by determining the average 0277. Then, the picture decoding apparatus 150 adds
value of the pixel values of co-located pixels of the respec Such predictive image to residual data generated by decod
tive reference blocks RB1 and RB2, and outputs the result ing the bit stream BS (Step S156).
ant as an interpolated block ReP.
0278 As described above, in the present embodiment,
0268 As described above, the contacts of the Switch 156 Since a picture with the Smallest reference indeX is Selected
are changed according to a prediction type. Therefore, when as a Second reference picture from among the pictures
a prediction type other than interpolative prediction is located after the current picture to be decoded in display
Selected, a block RB from the multi frame buffer 158 is order, it is possible to correctly decode the bit stream which
transmitted to the adder 155 via the Switch 156 as a has been coded by the picture coding apparatus according to
predictive image Signal Pre, whereas when interpolative the present invention described in the first embodiment.
prediction is Selected as a prediction type, an interpolated Moreover, it is possible to obtain the same effect as is
block ReP from the pixel interpolation unit 157 is transmit described in the first embodiment.
ted to the adder 153 via the Switch 156 as a predictive image
Signal Pre. 0279) Note that the bit stream BS includes the coded
0269. Here, the picture selection unit 159 in the present remapping information 27 as described in the first embodi
embodiment Selects, from among pictures Stored in the multi ment, and the multi frame buffer 158 specifies pictures
frame buffer 158, a picture to be used for direct mode as the indicated by the first reference indices and Second reference
Second reference picture. indices, based on remapping information obtained by decod
ing Such coded remapping information 27.
0270 More specifically, the picture selection unit 159
Selects a picture with the Smallest Second reference indeX as 0280 (Third Embodiment)
the Second reference picture for direct mode, from among 0281. The following describes the picture coding appa
the Subsequent pictures of the current picture to be decoded ratus according to the third embodiment of the present
in display order. invention with reference to the figures.
0271 Then, the picture selection unit 159 outputs, to the 0282 FIG. 17 is a block diagram showing the picture
vector buffer 160, the second reference index rRI2 of Such coding apparatus according to the third embodiment.
Selected Second reference picture. Note that the picture
selection unit 159 can know about the display order of each 0283 Such picture coding apparatus 200, which obtains
picture by using information about display order attached to a picture signal Img with the contents representing pictures
each picture. and codes Such picture Signal Img on a block-by-block basis,
0272 Accordingly, the picture decoding apparatus 150 is comprised of the Subtracter 101, the image decoding unit
according to the present embodiment can correctly decode 104, a variable length coding unit 203, the motion estimation
the bit stream BS outputted by the picture coding apparatus unit 109, the prediction type selection unit 108, the Switches
100 according to the first embodiment. More specifically, it 111 and 112, the pixel interpolation unit 106, the direct mode
is possible for the picture decoding apparatus 150 to decode processing unit 110, the vector buffer 114, the multi frame
the bit Stream outputted by the picture coding apparatus that buffer 107, and a picture selection unit 213.
Selects, from among pictures located after the current picture 0284. Here, the subtracter 101, the image decoding unit
to be coded in display order, a picture with the Smallest 104, the motion estimation unit 109, the prediction type
Second reference indeX as the Second reference picture to be selection unit 108, the Switches 111 and 112, the pixel
used for direct mode. interpolation unit 106, the direct mode processing unit 110,
0273 FIG. 16 is a flowchart showing the operation of the multi frame buffer 107, and the vector buffer 114 in the
decoding a picture in direct mode in the present embodi picture coding apparatus 200 respectively have the same
ment. functions as those of the picture coding apparatus 100
0274 First, the picture decoding apparatus 150 causes the according to the first embodiment.
picture selection unit 159 to select a picture with the smallest 0285) Stated another way, the present embodiment is
Second reference index value as the Second reference pic characterized by the functions of the picture Selection unit
ture, from among the Subsequent pictures of the current 213 and the variable length coding unit 203.
picture to be decoded in display order (Step S150). 0286 The picture selection unit 213 selects a picture
0275 Next, the picture decoding apparatus 150 causes located after the current picture to be coded in display order
the vector buffer 160 to specify, as a scaling vector rMV, the as a Second reference picture used for direct mode, from
first motion vector of a block that is co-located with the among pictures stored in the multi frame buffer 107.
current block to be decoded and included in the Second
reference picture Selected in the above manner, and to 0287. Then, the picture selection unit 213 outputs the
Specify, as a first reference picture, a picture referred to by Second reference indeX rRI2 indicating Such Selected Second
such first motion vector MV1 (Step S152). reference picture to the vector buffer 114, the direct mode
0276 Then, the picture decoding apparatus 150 generates processing unit 110, and the variable length coding unit 203.
a predictive image in direct mode through the direct mode 0288. Note that the picture selection unit 213 can know
processing unit 154, the multi frame buffer 158, and the about the display order of each picture by using information
pixel interpolation unit 157, utilizing the Second reference about display order attached to each picture. Furthermore,
picture Selected in the above manner as well as the first the picture Selection unit 213 Selects one Second reference
reference picture and the Scaling vector rMV Specified in the picture used in direct mode for the current picture to be
above manner (Step S154). coded. In other words, the same Second reference picture is
US 2004/0234143 A1 Nov. 25, 2004

selected for all the blocks included in the current picture to 0297. In other words, each header 41 in the bit stream
be coded to which direct mode shall be applied. BS1 includes not only the first time difference information
0289 Here, when remapping takes place in the present 24, the second time difference information 25, the time
embodiment, the picture selection unit 213 specifies the first difference information 26, and the coded remapping infor
reference indeX and the Second reference indeX assigned to mation 27, but also second index information 42 obtained by
each picture in the multi frame buffer 107 based on remap coding the above-described second reference index rRI2
ping information, as in the case of the first embodiment. outputted from the picture selection unit 213.
0290 For example, when remapping is not carried out, as 0298) Note that, as described above, the same value is
shown in FIG. 13A, the picture selection unit 213 selects, as used as a second reference index rRI2 for all blocks within
the Second reference picture for the current picture to be the current picture to be coded to which direct mode shall be
coded in direct mode, any one of the Subsequent pictures of applied. Therefore, a second reference index rRI2 shall be
the current block to be coded, i.e. any one of the picture with coded only once per picture. Also, the picture decoding
the picture number “15”, the picture with the picture number apparatus that decodes the bit Stream BS1 can uniquely
“13', and the picture with the picture number “11”. Then, Select Such Second reference picture used for direct mode, by
the picture Selection unit 213 outputs the Second reference obtaining Such Second reference indeX rRI2 and the remap
indeX rRI2 indicating Such Selected picture. For example, ping information.
when selecting the picture with the picture number “15” as 0299 Also, when not a direct mode scaling coefficient SP
the Second reference picture, the picture Selection unit 213 but display time information is used for processing per
outputs the Second reference indeX rRI2 indicating 0. formed by the direct mode processing unit 110, information
0291 Meanwhile, when remapping is carried out, as obtained by coding the display time information of the
shown in FIG. 13B, the picture selection unit 213 selects, as picture is included in the header 41 instead of the first time
the Second reference picture for the current picture to be difference information 24 and the second time difference
coded in direct mode, any one of the Subsequent pictures of information 25, as well as the time difference information
the current block to be coded, i.e. any one of the picture with 26.
the picture number “15”, the picture with the picture number 0300 FIG. 19 is a flowchart showing the operation of
“13', and the picture with the picture number “11”, as in the coding a picture in direct mode in the present embodiment.
above case. Then, the picture Selection unit 213 Specifies the
Second reference index rRI2 indicating Such selected picture 0301 First, the picture coding apparatus 200 causes the
based on the remapping information, and outputs it. picture Selection unit 213 to Select any one of the pictures
0292 Accordingly, in the present embodiment, it is pos located after the current picture to be coded in display order,
Sible to improve coding efficiency, even when remapping as the second reference picture (Step S200).
takes place, Since a picture that comes after the current 0302) Next, the picture coding apparatus 200 causes the
picture to be coded in display order is used as the Second vector buffer 114 to specify, as a scaling vector rMV, the first
reference picture at the time of coding in direct mode, as in motion vector MV1 of a block that is co-located with the
the case of the first embodiment. current block and included in the Second reference picture
0293. The variable length coding unit 203 performs vari Selected in the above manner, and to Specify, as a first
able length coding on the coded residual signals ER, the first reference picture, a picture referred to by Such first motion
reference indices RI1, the second reference indices RI2, the vector MV1 (Step S202).
first motion vectors MV1, the second motion vectors MV2, 0303. Then, the picture coding apparatus 200 generates a
the direct mode Scaling coefficients SP, the type information predictive image in direct mode through the direct mode
PT, and the remapping information, as well as on each processing unit 110, the multiframe buffer 107, and the pixel
Second reference indeX rRI2 indicating a Second reference interpolation unit 106, utilizing the Second reference picture
picture Selected by the picture Selection unit 213, and Selected in the above manner as well as the first reference
outputs the resultant as a bit stream BS1. In the case where picture and the scaling vector rMV specified in the above
the direct mode processing unit 110 does not use the direct manner (Step S204).
mode Scaling coefficients SP but display time information,
the variable length coding unit 203 performs variable length 0304. After generating the predictive image, the picture
coding on the display time information used for the above coding apparatus 200 generates a prediction error from the
performed Scaling without variable length coding the direct current block to be coded and the predictive image (Step
mode Scaling coefficients SP, and incorporates the variable S206), and then codes Such generated prediction error, the
length-coded display time information into the bit Stream Second reference indeX rRI2 indicating the Second reference
BS1 picture, and the remapping information (Step S208).
0294 FIG. 18 is a diagram showing the structure of a 0305 As described above, in the present embodiment,
header of the bit stream BS1. any one of the reference pictures located after the current
0295) The bit stream BS1 is structured on a picture-by picture to be coded in display order is Selected as the Second
picture basis, where information about the header 41 and reference picture in direct mode, and the Second reference
blockS is included on a picture-by-picture basis, as in the indeX rRI2 of Such Selected reference picture is coded and
case of the bit stream BS in the first embodiment. stored into the bit stream BS1.
0296. Here, the header 41 included in the bit stream BS1 0306 In the first embodiment, a picture with the smallest
on a picture-by-picture basis is different from the header 21 Second reference indeX is Selected as a Second reference
included in the bit stream BS on a picture-by-picture basis picture used for direct mode, from among the pictures
in the first embodiment. located after the current picture to be coded in display order.
US 2004/0234143 A1 Nov. 25, 2004

0307 Stated another way, in the third embodiment, as in 0315 Such picture decoding apparatus 250, which
the case of the first embodiment, it is possible to improve decodes the bit stream BS1 outputted from the picture
coding efficiency by Selecting, as the Second reference coding apparatus 200 in the third embodiment, is comprised
picture in direct mode, a picture that comes after the current of a variable length decoding unit 251, the image decoding
picture to be coded in display order. unit 152, the adder 153, the Switches 155 and 156, the direct
0308 The difference between the first embodiment and mode processing unit 154, the pixel interpolation unit 157,
the third embodiment is as described below. In the first the multi frame buffer 158, and the vector buffer 160.
embodiment, a Second reference picture in direct mode is 0316 Here, the image decoding unit 152, the adder 153,
determined mutually and uniquely between the picture cod the Switches 155 and 156, the direct mode processing unit
ing apparatus 100 and the picture decoding apparatus 150 by 154, the pixel interpolation unit 157, the multi frame buffer
having the picture coding apparatus 100 hold the procedure 158, and the vector buffer 160 of the image decoding unit
of determining a Second reference picture in direct mode as 250 respectively have the same functions as those of the
well as by having the picture decoding apparatus 150 in the picture decoding apparatuS 150 in the Second embodiment.
second embodiment hold the same procedure. In the third 0317. In other words, the present embodiment is charac
embodiment, on the other hand, a Second reference picture terized by the function of the variable length decoding unit
in direct mode is determined mutually and uniquely between 251.
the picture coding apparatus 200 and the picture decoding
apparatus by incorporating into a bit Stream information for 0318. The variable length decoding unit 251 obtains the
uniquely determining Such Second reference picture and by bit Stream BS1 from outside the picture decoding apparatus
having the picture coding apparatus 200 and the picture 250, performs variable length decoding on such bit stream
decoding apparatus that decodes Such bit stream perform BS1, and outputs the coded residual signals ER, the motion
processing based on Such information. vectors MV1 and MV2, the reference indices RI1 and RI2,
0309 AS described above, according to the present the direct mode Scaling coefficients SP, and the type infor
embodiment, it is possible to ensure consistency between mation PT. Here, when the bit stream BS1 does not include
coding processing and decoding processing by coding infor coded direct mode scaling coefficients SP but coded display
mation to identify a Second reference picture Selected in time information, the variable length decoding unit 251
coding processing. outputs the display time information instead of the direct
mode scaling coefficients SP.
0310 Note that in the present embodiment, any one of the 0319 Furthermore, the variable length decoding unit 251
pictures located after the current picture to be coded in decodes the Second indeX information 42 included in each
display order is Selected as a Second reference picture, but a
picture with the Smallest Second reference indeX may be header 41 in the bit stream BS1, and outputs the second
reference indices rRI2 to the vector buffer 160 and the direct
Selected as a Second reference picture from among the mode processing unit 154. Moreover, when each header 41
Subsequent pictures in display order, as in the case of the first
embodiment. Furthermore, the picture selection unit 213 includes coded remapping information 27, the variable
may Select the first reference picture of a current picture to length decoding unit 251 decodes the coded remapping
be coded used in direct mode. In this case, the first reference information 27, So as to generate remapping information.
index rRI1 and the second reference index rRI2 shall be Then, the variable length decoding unit 251 causes the multi
coded to be incorporated into the header 41 shown in FIG. frame buffer 158 to specify a picture indicated by each first
5. Moreover, the picture decoding apparatus that decodes the reference index RI1 (rRI1) and second reference index RI2
bit stream BS1 can identify the first reference picture and the (rRI2), based on the remapping information.
Second reference picture used in direct mode by obtaining 0320. The vector buffer 160 obtains, from the variable
their first reference index rRI1 and second reference index length decoding unit 251, the Second reference indeX rRI2,
rRI2 from the bit stream BS1. and Specifies, as the Second reference picture of the current
0311 What is more, according to the format of the bit picture to be decoded, a picture indicated by Such Second
stream BS1 in the present embodiment, the second index reference index rRI2. Then, the vector buffer 160 outputs, as
information 42 obtained by coding a Second reference indeX a scaling vector rMV, the first motion vector MV1 of a
rRI2 is included in the header 41 on a picture-by-picture predetermined block included in the Second reference pic
basis. However, the second reference information 42 may be ture, and outputs the first reference indeX RI1 indicating a
Stored, for example, in (i) the header of a coding unit Smaller picture referred to by such scaling vector rMV as a first
than a picture such as an MPEG slice, (ii) the header of a reference index rRI1.
coding unit made up of plural pictures Such as a GOP of 0321 AS described above, in the present embodiment, the
MPEG, or (iii) the header of the whole sequence. Moreover, Second reference picture used in direct mode is uniquely
in the case where a first reference picture is Selected in the determined from among the pictures located after the current
above-described manner, information obtained by coding a picture to be decoded in display mode, according to each
first reference index rRI1 indicating such first reference Second indeX information 42 and each coded remapping
picture shall be stored in a header described above. information 27 included in the bit stream BS1.
0312 (Fourth Embodiment) 0322 Here, a description is given of the operation of
0313 The following describes the picture decoding appa decoding a picture in direct mode in the present embodi
ratus according to the fourth embodiment of the present ment.
invention with reference to the figures. 0323 FIG. 21 is a flowchart showing the operation of
0314 FIG. 20 is a block diagram showing the picture decoding a picture in direct mode in the present embodi
decoding apparatus according to the fourth embodiment. ment.
US 2004/0234143 A1 Nov. 25, 2004

0324 First, the picture decoding apparatus 250 causes the scaling vector rMV and the first reference index rRI1
variable length decoding unit 251 to perform variable length corresponding to such second reference index rRI2"O”.
decoding on the bit Stream BS1, So as to obtain the Second 0335) Moreover, the multiframe buffer 303 in the present
reference index rRI2 (Step S250). When the bit stream BS1 embodiment has the function equivalent to that of the multi
includes the coded remapping information 27, the picture frame buffer 107 in the first and third embodiments, but
decoding apparatuS 250 obtains remapping information by information which it stores is under the management of the
decoding Such coded remapping information 27. management unit 3.04.
0325 Next, the picture decoding apparatus 250 causes 0336 Furthermore, the multi frame buffer 303 has a
the vector buffer 160 to specify, as a scaling vector rMV of Short-term memory and a long-term memory, where pictures
the current block to be decoded, the first motion vector of a are appropriately Stored into either the short-term memory or
block that is co-located with Such current block and included
the long-term memory.
in the Second reference picture indicated by the Second
reference indeX rRI2, and to Specify, as the first reference 0337 The short-term memory is a first-in-first-out
picture of the current picture to be decoded, a picture (FIFO) memory. When a new signal is stored into this
referred to by such first motion vector MV1 (Step S252). Short-term memory, its Storage contents are deleted in
0326. Then, the picture decoding apparatus 250 causes decreasing order of time Stored in the memory, So that a
certain number of the latest pictures are always Stored in
the direct mode processing unit 154, the multi frame buffer Such short-term memory as reference pictures.
158, and the pixel interpolation unit 157 to generate a
predictive image in direct mode, utilizing the Second refer 0338. The long-term memory is a random access memory
ence picture indicated by the Second reference indeX rRI2 capable of Storing reference pictures in an arbitrary area as
obtained in the above manner, as well as the first reference well as of reading out reference pictures Stored in an
picture and the scaling vector rMV specified in the above arbitrary area. Such long-term memory, which Stores pic
manner (Step S254). tures to be referred to for a longer period of time (e.g.
background pictures and pictures before Scene changes
0327. Then, the picture decoding apparatus 250 adds occur), Stores pictures equivalent to a longer period of time
Such predictive image to residual data generated by decod than those Stored in the short-term memory. Also, pictures
ing the bit stream BS1 (Step S256). are Stored into the long-term memory by moving pictures
0328. As described above, in the present embodiment, Stored in the short-term memory into the long-term memory.
Since a Second reference picture in direct mode is Specified 0339. The management unit 304 manages information
based on each Second indeX information 42 and remapping stored in the multiframe buffer 303 in the above manner, and
information 41 included in the bit stream BS1, it is possible causes remapping to take place when a management result
to correctly decode the bit stream BS1 transmitted from the indicates the need for remapping. In other words, the man
picture coding apparatus 200 according to the third embodi agement unit 304 generates remapping information, and
ment. Moreover, it is possible to obtain the Same effect as is outputS Such remapping information to the prediction type
described in the third embodiment. Selection unit 301, the motion estimation unit 302, the
0329 (Fifth Embodiment) variable length coding unit 305, and the multi frame buffer
303.
0330. The following describes the picture coding appa
ratus according to the fifth embodiment of the present 0340 For example, when judging that there has been a
invention with reference to the figures. change in the Scene (hereinafter referred to as “Scene
change') based on a result of obtaining Scene change
0331 FIG. 22 is a block diagram showing the picture information to be descried below, the management unit 304
coding apparatus according to the fifth embodiment. generates remapping information.
0332 Such picture coding apparatus 300, which obtains 0341 The prediction type selection unit 301, which has
a picture Signal Img with the contents representing pictures the same function as that of the prediction type Selection unit
and codes Such picture Signal Img on a block-by-block basis, 101 in the first and third embodiments, specifies the first
is comprised of the Subtracter 101, the image decoding unit reference indeX and the Second reference indeX assigned to
104,the Switches 111 and 112, the pixel interpolation unit each reference picture based on the remapping information,
106, the direct mode processing unit 110, the vector buffer when obtaining Such remapping information from the man
114, as in the case of the picture coding apparatus 100 agement unit 3.04.
according to the first embodiment, and further includes a
management unit 304, a motion estimation unit 302, a 0342. The motion estimation unit 302, which has the
same function as that of the motion estimation unit 102 in
prediction type selection unit 301, a multiframe buffer 303,
and a variable length coding unit 305. the first and third embodiments, specifies the first reference
indeX and the Second reference indeX assigned to each
0333 Here, the subtracter 101, the image decoding unit reference picture based on the remapping information, when
104, the Switches 111 and 112, the pixel interpolation unit obtaining Such remapping information from the manage
106, the direct mode processing unit 110, and the vector ment unit 3.04.
buffer 114 in the picture coding apparatus 300 respectively 0343 Stated another way, when not obtaining remapping
have the Same functions as those of the picture coding information from the management unit 304, the prediction
apparatus 100 in the first embodiment.
type selection unit 301 and the motion estimation unit 302
0334. Here, the vector buffer 114 obtains a second ref assign numerical values to the respective reference pictures
erence indeX rRI2 that always indicates 0, and outputs the preceding and Subsequent to the current picture to be coded,
US 2004/0234143 A1 Nov. 25, 2004
22

according to the default order, i.e. the order shown in FIG. 0348 Here, the management unit 304 may generate
13A, So as to Specify the first reference indices and the remapping information that does not change the Second
Second reference indices. Meanwhile, when obtaining reference index “0” of the reference picture which shall be
remapping information from the management unit 304, the assigned Such Second reference indeX 0 by default.
prediction type selection unit 301 and the motion estimation 0349 This enables the second reference index 0 to be
unit 302 assign numerical values to the respective reference always assigned to a reference picture that is located after
pictures preceding and Subsequent to the current picture to
be coded, according to the order to be determined by taking the current picture to be coded in display order, even when
into account the remapping information to the default order, remapping occurs.
So as to specify the first reference indices and the Second 0350 Referring to FIG. 23, a description is given of
reference indices. remapping performed by the management unit 304 with the
0344) The variable length coding unit 305 performs vari above configuration.
able length coding on the coded residual signals ER, the first 0351 FIG. 23 is a diagram explaining remapping per
reference indices RI1, the second reference indices RI2, the formed by the management unit 3.04.
first motion vectors MV1, the second motion vectors MV2,
the direct mode Scaling coefficients SP, and the type infor 0352 (a) in FIG. 23 is a diagram showing the arrange
mation PT, so as to output the resultant as a bit stream BS2, ment of the current picture to be coded and reference
as in the case of the variable length coding unit 105 in the pictures in display order.
first and third embodiments. Moreover, when obtaining 0353 When a current picture TP, which is a B picture, is
remapping information from the management unit 304, the coded, any of the Ppictures preceding or Subsequent to Such
variable length coding unit 305 codes Such remapping current picture to be coded TP, e.g. reference pictures RP1,
information, and incorporates Such coded remapping infor RP2, RP3, RP4 and RP5, is used. Moreover, the reference
mation into each header of the bit stream BS2. In the case pictures RP2, RP3, RP4, and RP5, for example, which are
where display time information is used instead of the direct located near the current picture TP are stored in the short
mode Scaling coefficients SP for the processing performed term memory in the multi frame buffer 303, whereas the
by the direct processing unit 110, the variable length coding reference picture RP1 located far away from the current
unit 305 performs variable length coding on the display time picture TP is stored in the long-term memory in the multi
information used for the above-performed Scaling, without frame buffer 303.
variable length coding the direct mode Scaling coefficients
SP, and incorporates Such variable length-coded display time 0354) In order to identify the reference pictures RP1,
information into the bit stream BS2. RP2, RP3, RP4 and RP5, first reference indices and second
0345 Here, when generating remapping information, the reference indices are assigned to these pictures as shown in
management unit 304 according to the present embodiment (b) in FIG. 23 by default.
generates Such remapping information as makes first refer 0355 As (b) in FIG.23 shows, the reference picture RP1
ence indices and Second reference indices assigned to the is assigned the first reference indeX “4” and the Second
respective reference pictures Satisfy a certain condition. reference index “4”, the reference picture RP2 is assigned
the first reference index “1” and the second reference index
0346. In the case where there is a scene change when the “3”, the reference picture RP3 is assigned the first reference
current block is to be coded by means of interpolative
prediction other than direct mode, the management unit 304 index “0” and the second reference index "2', the reference
generates Such remapping information as allows the first picture RP4 is assigned the first reference index “2” and the
reference index of 0 to be assigned preferentially to a second reference index “0”, and the reference picture RP5 is
reference picture Stored in the long-term memory. The assigned the first reference index “3” and the second refer
ence index “1”.
judgment on whether a Scene change occurs or not is made
outside the picture coding apparatus 300 on the basis of the 0356. Here, in the case where the current picture TP is to
input picture Signal Img, So that the result of Such judgment be coded by means of interpolative prediction other than in
shall be provided to the management unit 304 as scene direct mode, the management unit 304 generates remapping
change information. Scene change detection methods (judg information that allows the first reference index 0 to be
ment methods) include the following method: determine an assigned from a reference picture Stored in the long-term
absolute value difference between each of pixels co-located memory, when making the following judgments: the Scene
with each other in temporally Successive pictures, So as to A similar to the scene A that includes the current picture TP
add absolute difference values of all the pixels in the is older than the Scene B which is not similar to the scene A';
pictures, and judge that there is a Scene change when Such and the reference picture that is located temporally and
addition value exceeds a predetermined threshold. Further immediately before the Scene A (being the reference picture
more, it is possible to code a frame (picture) detected as a RP3 in this case) is included in the scene B which is not
Scene change as an I picture. Accordingly, it becomes Similar to the Scene A, i.e. a Scene change occurs after the
possible to Search out the point of a Scene change by reference picture that is located temporally and immediately
Searching for an I picture. before the scene A.
0347. Furthermore, when the current block is coded by 0357. In other words, as shown in (c) in FIG. 23, the first
means of interpolative prediction in direct mode, the man reference index 0 is assigned to the reference picture RP1
agement unit 304 generates Such remapping information as Stored in the long-term memory.
allows the Second reference index of 0 to be assigned to a
reference picture located after Such current picture to be 0358 In the present embodiment, as described above,
coded in display order. when a Scene change occurs at the time of coding the current
US 2004/0234143 A1 Nov. 25, 2004
23

picture by means of interpolative prediction other than in 0367 (Variation)

direct mode, remapping information is generated that 0368 Here, a description is given of a variation of the
enables the first reference index 0 to be preferentially management unit according to the present embodiment.
assigned to a reference picture Stored in the long-term
memory. Accordingly, the first reference picture with the 0369 FIG. 25 is a diagram showing a configuration of a
first reference index “0” used for interpolative prediction has picture coding apparatuS 300' according to the variation of
picture information much similar to the current picture to be the present embodiment.
coded. This consequently contributes to the improvement of 0370. The picture coding apparatus 300' according to the
coding efficiency.
variation is comprised of the image coding unit 102, the
0359 Stated another way, as shown in (b) in FIG. 23, in image decoding unit 104, a management unit 304' and
the case where Such remapping information is not to be others, as in the case of the aforementioned picture coding
generated, the reference picture RP3 indicated by the first apparatus 300.
reference index “0” shall be used for coding the current
picture TP. However, since the reference picture RP3 is 0371. Here, in the present embodiment, the second ref
dissimilar to the current picture to be coded TP because the erence index 0 shall be assigned to a reference picture that
reference picture RP3 is included in a different Scene, coding is located after the current picture to be coded in display
efficiency becomes lowered. order, when Such current picture is coded by means of
0360 Meanwhile, when the current picture TP is coded in interpolative prediction in direct mode. Stated another way,
direct mode, the management unit 304 generates remapping the management unit 304 prohibits the assignment of the
information as shown in (d) in FIG. 23 that allows the Second reference index 0 to any reference pictures that have
Second reference index 0 to be assigned to a reference been located before the current picture in display order. AS
picture that comes after the current picture to be coded TP a result, no first motion vectors used for coding blocks in a
in display order, e.g. the reference picture RP5. Furthermore, reference picture that has been located before a current
the management unit 304 may generate remapping informa picture in display order, shall be used as Scaling vectors.
tion as shown in (b) in FIG. 23 that does not allow the 0372 Thus, the present variation can improve coding
second reference index “0”, which shall be assigned to the efficiency by omitting processing for Storing, into the Vector
reference picture RP4 by default, to be assigned to another buffer 114, the first motion vectors of blocks in a reference
reference picture. picture that has been located before a current picture in
0361. As described above, in the present embodiment, display order for use in the coding of Such current picture.
when the current block is coded by interpolative prediction In other words, in the present variation, the management unit
in direct mode, the Second reference index 0 is assigned to 304' in the picture coding apparatus 300' deletes, from the
a reference picture located after the current picture to be vector buffer 114, the first motion vectors of a reference
coded in display order, So as to use Such reference picture picture that has been located before the current picture in
with the second reference index “0” as the second reference display order.
picture. Accordingly, it becomes possible to improve coding 0373 This prevents motion vectors not necessary for
efficiency. coding from being stored in the vector buffer 114, and
0362 FIG. 24 is a flowchart showing an example opera therefore reduces the amount of memory of the vector buffer
tion performed in the present embodiment. 114.
0363 First, the picture coding apparatus 300 judges 0374 (Sixth Embodiment)
whether or not there is a Scene change, according to whether
or not the management unit 304 has obtained the above 0375. The following describes the picture decoding appa
mentioned scene change information (Step S300). ratus according to the Sixth embodiment of the present
0364. Here, when judging that there is a scene change (Y invention with reference to the figures.
in Step S300), the picture coding apparatus 300 performs 0376 FIG. 26 is a block diagram showing the picture
remapping in a manner in which the following conditions decoding apparatus according to the Sixth embodiment.
are satisfied (Step S302): the first reference index 0 is 0377 Such picture decoding apparatus 350, which
assigned to a reference picture Stored in the long-term decodes the bit stream BS2 outputted from the picture
memory; and the Second reference index 0 is assigned to any coding apparatus 300' according to the variation of the fifth
one of the reference pictures located after the current picture embodiment, is comprised of the variable length decoding
to be coded. Meanwhile, when judging that there is no Scene unit 151, the image decoding unit 152, the adder 153, the
change (N in Step S300), the picture coding apparatus 300 Switches 155 and 156, the direct mode processing unit 154,
assigns first reference indices and Second reference indices the pixel interpolation unit 157, the multiframe buffer 158,
according to the default, without performing remapping and the vector buffer 160, as in the case of the second
(Step S304). embodiment, and further includes a management unit 354.
0365 Next, the picture coding apparatus 300 specifies the
first reference picture and the Second reference picture based 0378. The picture decoding apparatus 350 has the same
on the first reference indices and Second reference indices configuration as the one described in the Second embodi
assigned in the above manner, and generates a predictive ment in which the picture selection unit 159 outputs, to the
image based on these reference pictures (Step S306). vector buffer 160, the second reference index rRI2 that
0366 Subsequently, the picture coding apparatus 300 always indicate 0.
generates a prediction error from the predictive image (Step 0379 Moreover, the variable length decoding unit 151
S308), and codes it (Step S310). obtains the bit stream BS2, and when Such bit stream BS2
US 2004/0234143 A1 Nov. 25, 2004
24

include remapping information, the variable length decoding 0392 FIG. 29 is a flowchart showing the process of
unit 151 causes the multi frame buffer 158 to specify a calculating the motion vectors MV01 and MV02.
picture indicated by each first reference index RI1 (rRI1) 0393 First, the picture coding apparatus 600 determines
and second reference index RI2 (rRI2), based on Such the block B1 on the first reference picture RP1 which is
remapping information.
co-located with the block Bn referred to by the motion
0380 Here, the management unit 354 monitors reference vector MV10 of the block B00 (Step S1).
pictures stored in the multiframe buffer 158, so as to delete,
from the vector buffer 160, the first motion vectors MV1 of 0394 Next, the picture coding apparatus 600 regards the
a reference picture that has been located before the current motion vector MV0 extending from the block B00 to the
picture to be decoded in display order. block B1 as a scaling vector (Step S2).
0381. This prevents motion vectors not necessary for 0395. Then, the picture coding apparatus 600 determines
decoding from being stored in the vector buffer 160 also in the motion vectors MV01 and MV02 from Such scaling
the present embodiment, as in the case of the variation of the vector MVO (Step S3).
fifth embodiment, and therefore reduces the amount of
memory of the vector buffer 160. 0396 AS described above, by using one picture as the
first reference picture to be commonly used in direct mode
0382 (Seventh Embodiment) regardless of a reference picture referred to by the motion
0383 FIG. 27 is a block diagram showing a picture vector MV10, it becomes not necessary for the vector buffer
coding apparatus according to the Seventh embodiment. 914 to store the picture number of a reference picture
referred to by the motion vector MV10 (being the picture
0384. Here, units and signals shown in FIG.27 which are number of the reference picture RPn in FIG. 28). Instead, as
the same as those of the existing picture coding apparatus shown in FIG. 27, the first reference picture selection unit
900 shown in FIG. 7 are assigned the same numbers as those 601 selects a first reference picture to be commonly used in
assigned to the units and Signals of the picture coding direct mode, and outputs the first reference index rRI1 of
apparatus 900, and therefore no description is given of them. Such first reference picture. Note that an example method of
0385) The present embodiment is characterized by that Selecting a first reference picture to be commonly used in
the first reference picture selection unit 601 selects one first direct mode is to Select a reference picture which comes
reference picture to be commonly used for coding the blockS before the current picture to be coded in display order and
in the current picture in direct mode. Accordingly, a prede which is assigned the smallest first reference index value.
termined reference picture shall be commonly used as the 0397 As described above, according to the present
first reference picture to be referred to by Scaling vectors in embodiment, by using one first reference picture as the
direct mode. picture to be commonly used for all the blockS in a picture
0386 FIG. 28 is a conceptual diagram showing direct to be coded in direct mode, there is no need to Store in the
mode according to the Seventh embodiment. memory information used to identify a reference picture
0387. In FIG. 28, the reference picture RP1 serves as the referred to by a Scaling vector. In other words, it becomes
first reference picture to be commonly used in direct mode. possible to reduce the amount of memory. Furthermore, by
In other words, the first motion vector MV10 of the block
using a single first reference picture as the picture to be
B00 refers to the reference picture RPn, but the first motion commonly used for all the blockS in a picture to be coded in
vector MVO1 of the block B0 to which direct mode is direct mode, it becomes possible to Simplify coding pro
applied refers to the above-described reference picture RP1. cessing and therefore to improve coding efficiency. Note that
information used to identify a picture referred to by a first
0388 More specifically, the motion vector MVO approxi motion vector is, for example, its picture number shown in
mately derived from the block B1 on the common reference FIG. 13, but any other information may serve as such
picture RP1 is used as a Scaling vector, So as to derive the identification information as long as a picture referred to by
motion vectors MV01 and MV02 used for coding the block the motion vector can be uniquely determined by Such
B0 in the current picture TP based on the scaling of Such information.
scaling vector. Stated another way, the motion vector MV01
of the block B0 on the current picture to be coded TP which 0398. Note that in the present embodiment, the picture
is a motion vector derived from the block RB01 on the with the Smallest first reference indeX among pictures
picture RP1 and the motion vector MV02 of the block RB02 located before the current picture to be coded is Selected as
on the picture RP3 are determined by using the motion the first reference picture to be commonly used in direct
vector MVO as a scaling vector. mode. However, the reference picture that is closest to the
current picture in display order among the pictures preced
0389) Note that in the present embodiment, the direct ing Such current picture may be selected as the above first
mode processing unit 910' performs the above-described reference picture.
processing in direct mode.
0390 Meanwhile, conventionally, information for iden 0399. Also, in the present embodiment, by using a pre
tifying the reference picture RPn to be refereed to by the ceding picture of the current picture as at least one of the
motion vector MV10 is required for performing scaling and reference pictures referred to by Such current picture to be
Such information needs to be stored on the memory. coded, which is a B picture, there arises no possibility that
only pictures located after the current picture to be coded
0391) Here, referring to FIG. 29, a description is given of will be used as reference pictures. Accordingly, it becomes
a method of calculating the motion vectors MV01 and possible to raise the possibility of improving coding effi
MVO2. ciency.
US 2004/0234143 A1 Nov. 25, 2004
25

0400 Furthermore, in the present embodiment, a second 0410 (Ninth Embodiment)

reference picture may be Selected in a manner equivalent to 0411 FIG. 31 is a block diagram showing a picture
that of the first embodiment. In this case, coding efficiency coding apparatus according to the ninth embodiment.
will be improved as a result of combing the coding method
presented in the first embodiment and the coding method 0412 Here, units and signals shown in FIG.31 which are
presented in the present embodiment. the same as those of the existing picture coding apparatus
900 shown in FIG. 7 are assigned the same numbers as those
04.01 Moreover, in the present embodiment, the direct assigned to the units and Signals of the picture coding
mode processing unit 910 performs Scaling on a Scaling apparatus 900, and therefore no description is given of them.
vector rMV by use of a direct mode scaling coefficient SP. 0413. In the seventh embodiment, the first reference
However, display time information may be used to perform picture to be commonly used in direct mode is Selected by
Scaling on the Scaling vector rMV, as in the case of the other the same Single method which has been predetermined by
embodiments. In such case, referring to FIG. 28, not the the picture coding apparatus 600 and the picture decoding
motion vector MVO but the motion vector MV10 shall be apparatuS 650, and information indicating Such Selected first
used as a Scaling vector. Stated another way, the first motion reference picture shall not be stored in the bit stream BS0.
vector MV01 of the current block B0 is determined by In the present embodiment, however, the first reference
scaling the motion vector MV10 according to the ratio of the index of the selected first reference picture shall be stored in
difference in the display times between the picture TP and a bit Stream.
the picture RPn with respect to the difference in the display
times between the picture RP3 and the picture RPn, and the 0414. In other words, the ninth embodiment is different
Second motion vector MV02 of the current block B0 is from the seventh embodiment in the point described below.
determined by scaling the motion vector MV10 according to In the seventh embodiment, the first reference picture in
the ratio of the difference in the display times between the direct mode is uniquely determined by having the picture
picture TP and the picture RP3 with respect to the difference coding apparatus 600 hold the procedure for uniquely deter
between the display times in the picture RP3 and the picture mining the first reference picture in direct mode as well as
RPn. by having the picture decoding apparatuS 650 in the eighth
embodiment hold the same procedure. In the ninth embodi
0402 (Eighth Embodiment) ment, on the other hand, by incorporating, into a bit Stream,
information used for uniquely determining the first reference
0403 FIG. 30 is a block diagram showing a picture picture in direct mode, such first reference picture in direct
decoding apparatus according to the eighth embodiment. mode is mutually and uniquely determined by the picture
04.04 Here, units and signals shown in FIG. 30 which are decoding apparatus that decodes Such bit Stream and by the
the same as those of the existing picture decoding apparatus picture coding apparatus 700.
950 shown in FIG. 9 are assigned the same numbers as those 0415. The first reference picture selection unit 701 selects
assigned to the units and Signals of the picture decoding a first reference picture for direct mode from among refer
apparatus 950, and no description is given of them. ence pictures, and outputs the first reference indeX rRI1 of
Such Selected reference picture to the direct mode processing
04.05 The picture decoding apparatus 650 according to unit 910' and the variable length coding unit 903.
the present embodiment decodes the bit stream BS0 output 0416) The variable length coding unit 903 performs vari
ted by the picture coding apparatus 600 described in the
Seventh embodiment. able length coding on each coded residual signal ER, first
reference index RI1, second reference indeX RI2, first
0406 A first reference picture selection unit 651 selects a motion vector MV1, Second motion vector MV2, direct
first reference picture in a manner which is the same as that mode scaling coefficient SP, type information PT, and first
of the first reference picture selection unit 601 shown in reference index rRI1 which is outputted by the first reference
FIG. 27. Then, the first reference picture selection unit 651 picture selection unit 701, and outputs the resultant as a bit
outputs, to the direct mode processing unit 954, the first stream BS3. In the case where display time information is
reference index rRI1 of such selected first reference picture. used by the direct mode processing unit 910' instead of the
direct mode Scaling coefficient SP, the variable length coding
04.07 Subsequently, the direct mode processing unit 954 unit 903 performs variable length coding on the display time
performs the processing in direct mode shown in FIG. 28. information used for the above-performed Scaling without
0408. As described above, by using one first reference variable length coding the direct mode Scaling coefficient SP,
picture which shall be commonly used for all the blocks in and incorporates the variable length-coded display time
information into the bit stream BS3.
a picture Subject to direct mode coding, it becomes unnec
essary to hold information used to identify reference pictures 0417 FIG. 32 shows the format of the bit stream BS3
for direct mode. This also contributes to the reduction in the according to the ninth embodiment.
amount of memory of the vector buffer 960 compared to the 0418. The header is the only difference between the
conventional cases. format of the bit stream BS3 and the format of the bit stream
04.09 AS described above, according to the present BS0 of the existing picture coding apparatus 900 shown in
F.G. 6.
embodiment, it is possible to correctly decode the bit Stream
BS0 which has been coded by the picture coding apparatus 0419. The same value is used as a first reference index
600 in the seventh embodiment. Moreover, it is possible to rRI1 for all blocks within a picture to which direct mode
obtain the same effect as is described in the seventh embodi shall be applied. Therefore, it is enough to code the first
ment. reference indeX rRI1 only once per picture. Such being the
US 2004/0234143 A1 Nov. 25, 2004
26

case, the variable length coding unit 903 codes the first 0425 (Tenth Embodiment)
reference indeX rRI1, So as to incorporate Such coded first 0426 FIG. 33 is a block diagram showing a picture
reference index rRI1 into each header 51 of the bit stream
BS3 as first index information 52. Note that the header 51
decoding apparatus according to the tenth embodiment.
may include display time information instead of the first 0427 Here, units and signals shown in FIG.33 which are
time difference information 4 and the second time difference the same as those of the existing picture decoding apparatus
information 5, as well as the time difference information 6 950 shown in FIG. 9 are assigned the same numbers as those
shown in FIG. 32. The picture decoding apparatus that assigned to the units and Signals of the picture decoding
decodes the bit stream BS3 can specify the first reference apparatus 950, and no description is given of them.
picture for direct mode by obtaining, from the bit Stream 0428 The picture decoding apparatus 750 according to
BS3, the first reference index rRI1 indicated by the first the present embodiment decodes the bit stream BS3 output
index information 52. ted by the picture coding apparatus 700 described in the
0420 AS described above, according to the present ninth embodiment, i.e. the bit Stream that includes informa
embodiment, by using the common first reference picture for tion indicating the first reference index of the first reference
coding all the blocks within a picture which shall be coded picture to be commonly used for direct mode.
in direct mode, it becomes unnecessary to Store information 0429 The variable length decoding unit 951 obtains the
used to identify a reference picture refereed to by a Scaling bit stream BS3 so as to perform variable length decoding on
vector. Stated another way, it becomes possible to reduce the it, and outputs the coded residual signals ER, the first
amount of memory of the vector buffer 914 compared with reference indices RI1, the second reference indices RI2, the
the conventional cases. Moreover, it is also possible to first motion vectors MV1, the second motion vectors MV2,
Simplify coding processing by using the common first ref the direct mode Scaling coefficients SP, the type information
erence picture for coding all the blocks within a picture PT, and the first reference index rRI1 for direct mode. Here,
which shall be coded in direct mode. Also, it is possible to when the bit stream BS3 includes not the coded direct mode
ensure consistency between coding processing and decoding Scaling coefficients SP but coded display time information,
processing by coding information indicating the first refer the variable length decoding unit 951 outputs the display
ence picture used for direct mode and by incorporating Such time information instead of the direct mode Scaling coeffi
coded information into the bit stream BS3. cients SP.
0421 Note that it is also possible to select a picture with 0430. The first reference index rRI1 for direct mode
the Smallest first reference indeX among pictures that are indicates the first reference picture to be commonly used for
located before the current picture to be coded in display direct mode. Thus, there is no need for the vector buffer 960
order, as the first reference picture to be commonly used in to hold information for identifying reference pictures used
direct mode. for direct mode, resulting in the reduction in the amount of
memory of the vector buffer 960.
0422. Also, in the present embodiment, by using a pre 0431 AS described above, according to the present
ceding picture of the current picture as at least one of the
reference pictures referred to by Such current picture to be embodiment, it is possible to correctly decode the bit Stream
coded, which is a B picture, there arises no possibility that BS3 outputted by the picture coding apparatus 700 in the
only pictures located after the current picture to be coded ninth embodiment. Furthermore, by using the Same refer
will be used as reference pictures. Accordingly, it becomes ence picture for all the blocks within a picture which are
possible to improve coding efficiency. coded in direct mode, it becomes unnecessary to Store, on
the memory, information for identifying a reference picture
0423 Furthermore, in the present embodiment, a second referred to by a Scaling vector. In other words, it becomes
reference picture may be Selected in a manner equivalent to possible to reduce the amount of memory of the vector
that of the third embodiment. In this case, the first index buffer 960 compared with the conventional cases. Moreover,
information 52 obtained by coding the first reference index it is possible to obtain the same effect as described in the
rRI1 and the second index information obtained by coding ninth embodiment.
the Second reference indeX rRI2 are incorporated into the 0432 (Eleventh Embodiment)
header 51 shown in FIG. 32. Moreover, the picture decoding
apparatus that decodes the bit stream BS3 can specify the 0433) If a program for realizing the picture coding
first reference picture and Second reference picture used for method or the picture decoding method as shown in each of
direct mode by obtaining the first reference index rRI1 the aforementioned embodiments is recorded on a recording
indicated by such first index information 52 and the second medium Such as a flexible disk, it becomes possible to easily
reference index rRI2 indicated by such second index infor perform the processing presented in each of the above
mation. embodiments in an independent computer System.
0424 What is more, according to the format of the bit 0434 FIGS. 34A, 34B, and 34C are diagrams explaining
stream BS3 in the present embodiment, the first index a recording medium that Stores a program for realizing the
information 52 obtained by coding the first reference index picture coding method and the picture decoding method
rRI1 is included in the header 51 on a picture-by-picture according to the first to tenth embodiments.
basis. However, the first index information 52 may be stored, 0435 FIG.34B shows an external view of a flexible disk
for example, in (i) the header of a coding unit Smaller than FD viewed from the front, a schematic cross-sectional view,
a picture Such as an MPEG slice, (ii) the header of a coding and a disk body FD1, while FIG. 34A illustrates an example
unit made up of plural pictures such a GOP (Group of physical format of the disk body FD1 as a recording medium
Picture) of MPEG, or (iii) the header of the whole sequence. itself.
US 2004/0234143 A1 Nov. 25, 2004
27

0436 The disk body FD1 is contained in a case F, and a the telephone network ex104, which enables live distribu
plurality of trackS Tr are formed concentrically on the tion or the like based on coded data transmitted by the user
surface of the disk body FD1 in the radius direction from the using the camera ex113. Either the camera ex113 or a server
periphery, each track being divided into 16 SectorS Se in the and the like capable of data transmission processing may
angular direction. Therefore, in the flexible disk FD storing code the shot data. Also, moving picture data shot by a
the above-mentioned program, the picture coding method or camera eX116 may be transmitted to the Streaming Server
the picture decoding method as Such program is recorded in ex103 via the computer ex111. The camera ex116 is a device
an area allocated for it on the disk body FD1. Such as a digital camera capable of shooting Still pictures and
0437. Meanwhile, FIG. 34C shows the structure for moving pictures. In this case, either the camera eX116 or the
recording and reading out the program on and from the computer eX111 may code the moving picture data. An LSI
flexible disk FD. ex117 included in the computer ex111 or the camera ex116
performs coding processing. Note that Software for coding
0438. When the program is recorded on the flexible disk and decoding pictures may be integrated into a certain type
FD, the picture coding method or the picture decoding of storage medium (such as a CD-ROM, a flexible disk and
method as the above program is written by the use of the a hard disk) that is a recording medium readable by the
computer system Cs via a flexible disk drive FDD. Mean computer eX111 and the like. Furthermore, the camera
while, when the picture coding method or the picture decod equipped cellular phone eX115 may transmit the moving
ing method is constructed in the computer System CS picture data. This moving picture data is data coded by an
through the program on the flexible disk FD, the program is LSI included in the cellular phone ex115.
read out from the flexible disk FD via the flexible disk drive
FDD and transferred to the computer system Cs. 0447. In the content supply system ex100, content (e.g. a
music live video) which has been shot by the user using the
0439. The above explanation is made on the assumption camera eX113, the camera eX116 or the like is coded in the
that a recording medium is a flexible disk FD, but an optical Same manner as the above-described embodiments and
disc may also be used. In addition, the recording medium is transmitted to the Streaming Server eX103, and the Streaming
not limited to this, and any other medium Such as an IC card server ex103 makes stream distribution of the content data
and a ROM cassette capable of recording a program can also to clients at their request. The clients here include the
be used. computer ex111, the PDA ex112, the camera ex113, the
0440 (Twelfth Embodiment) cellular phone eX114 and So forth capable of decoding the
above coded data. The content Supply System ex100 with the
0441 The following explains applications of the picture above configuration is a System that enables the clients to
coding method and the picture decoding method as shown in receive and reproduce the coded data and realizes personal
the above embodiments as well as a System using them. broadcasting by allowing them to receive, decode and repro
duce the data in real time.
0442 FIG. 35 is a block diagram showing an overall
configuration of a content Supply System eX100 for realizing 0448. The picture coding apparatus and the picture
a content distribution Service. The area for providing a decoding apparatus presented in the above embodiments can
communication Service is divided into cells of desired size, be used for coding and decoding to be performed in each of
and base stations ex107-ex110, which are fixed wireless the devices making up the above System.
Stations, are placed in the respective cells. 0449 An explanation is given of a cellular phone as an
0443) In this content supply system ex100, devices such example.
as a computer ex111, a PDA (Personal Digital Assistant) 0450 FIG. 36 is a diagram showing the cellular phone
eX112, a camera eX113, a cellular phone eX114, and a ex115 that employs the picture coding method and the
camera-equipped cellular phone eX115 are respectively con picture decoding method explained in the above embodi
nected to the Internet ex101 via an Internet service provider ments. The cellular phone ex115 has an antenna ex201 for
ex102, a telephone network ex104, and the base stations transmitting/receiving radio waves to and from the base
eX107 eX110. station eX110, a camera unit eX203 Such as a CCD camera
0444. However, the content Supply system ex100 is not capable of Shooting Video and Still pictures, a display unit
limited to the combination as shown in FIG. 35, and may be eX202 Such as a liquid crystal display for displaying the data
connected to a combination of any of them. Also, each of the obtained by decoding Video and the like Shot by the camera
devices may be connected directly to the telephone network unit ex203 and video and the like received by the antenna
ex104, not via the base stations ex107-ex110, which are eX201, a main body including a group of operation keys
fixed wireleSS Stations. ex204, a voice output unit ex208 Such as a speaker for
0445. The camera ex113 is a device such as a digital outputting voices, a Voice input unit eX205 Such as a
Video camera capable of Shooting moving pictures. The microphone for inputting voices, a recording medium eX207
cellular phone may be a cellular phone of a PDC (Personal for Storing coded data or decoded data Such as data of
Digital Communication) system, a CDMA (Code Division moving or Still pictures shot by the camera, data of received
Multiple Access) system, a W-CDMA (Wideband-Code e-mails and moving picture data or Still picture data, and a
Division Multiple Access) system or a GSM (Global System slot unit ex206 for enabling the recording medium ex207 to
for Mobile Communications) system, a PHS (Personal be attached to the cellular phone ex115. The recording
Handyphone system) or the like, and may be any one of medium ex207 is embodied as a flash memory element, a
these. kind of EEPROM (Electrically Erasable and Programmable
Read Only Memory) that is an electrically erasable and
0446. Furthermore, a streaming server ex103 is con rewritable nonvolatile memory, Stored in a plastic case Such
nected to the camera eX113 via the base station eX109 and as an SD card.
US 2004/0234143 A1 Nov. 25, 2004
28

0451 Next, referring to FIG. 37, an explanation is given eX203 is taking place, to the multiplexing/demultiplexing
of the cellular phone ex115. In the cellular phone ex115, a unit ex308 as digital voice data via the Voice processing unit
main control unit ex311 for centrally controlling the display ex305.
unit ex202 and each unit of the main body having the 0457. The multiplexing/demultiplexing unit ex308 mul
operation keyS eX204 is configured in a manner in which a tiplexes the coded picture data Supplied from the picture
power Supply circuit unit eX310, an operation input control coding unit ex312 and the Voice data Supplied from the Voice
unit ex304, a picture coding unit ex312, a camera interface processing unit ex305 using a predetermined method, the
unit ex303, an LCD (Liquid Crystal Display) control unit modem circuit unit ex306 performs spread Spectrum pro
ex302, a picture decoding unit ex309, a multiplexing/de cessing on the resulting multiplexed data, and the transmit/
multiplexing unit ex308, a recording/reproducing unit receive circuit unit ex301 performs digital-to-analog con
ex307, a modem circuit unit ex306, and a voice processing version processing and frequency transformation processing
unit ex305 are interconnected via a synchronous bus ex313. on the resultant, So as to transmit the processed data via the
0452. When a call-end key or a power key is turned on by antenna ex201.
the user's operation, the power Supply circuit unit eX310 0458 When receiving, in data communication mode,
Supplies each unit with power from a battery pack, So as to data included in a moving picture file which is linked to a
activate the camera-equipped digital cellular phone eX115 to Web page or the like, the modem circuit unit ex306 performs
make it into a ready State. inverse spread spectrum processing on the received data
0453. In the cellular phone ex115, the voice processing received from the base station ex110 via the antenna eX201,
unit ex305 converts a voice signal received by the voice and sends out the resulting multiplexed data to the multi
input unit eX205 in conversation mode into digital voice data plexing/demultiplexing unit ex308.
under the control of the main control unit ex311 comprised 0459. In order to decode the multiplexed data received
of a CPU, a ROM, a RAM and others, the modem circuit via the antenna eX201, the multiplexing/demultiplexing unit
unit ex306 performs spread spectrum processing on it, and ex308 separates the multiplexed data into a bit stream of
a transmit/receive circuit unit ex301 performs digital-to picture data and a bit Stream of Voice data, and Supplies Such
analog conversion processing and frequency transformation coded picture data to the picture decoding unit ex309 and
processing on the data, So as to transmit the resultant via the such voice data to the voice processing unit ex305 via the
antenna ex201. Also, in the cellular phone ex115, data synchronous bus ex313.
received by the antenna ex201 in conversation mode is
amplified and performed of frequency transformation pro 0460 Next, the picture decoding unit ex309, which
cessing and analog-to-digital conversion processing, the includes the picture decoding apparatus according to the
modem circuit unit ex306 performs inverse spread spectrum present invention in its configuration, decodes the bit stream
processing on the resultant, and the Voice processing unit of the picture data using the decoding method paired with
eX305 converts it into analog Voice data, So as to output it via the coding method shown in the above-mentioned embodi
the voice output unit ex208. ments So as to generate moving picture data for reproduc
0454. Furthermore, when sending an e-mail in data com tion, and Supplies Such data to the display unit eX202 via the
munication mode, text data of the e-mail inputted by oper LCD control unit ex302. Accordingly, moving picture data
ating the operation keyS eX204 on the main body is Sent out included in the moving picture file linked to a Web page, for
to the main control unit ex311 via the operation input control instance, is displayed. At the same time, the Voice processing
unit ex304. In the main control unit ex311, after the modem unit ex305 converts the voice data into analog voice data,
circuit unit eX306 performs spread spectrum processing on and then supplies this to the voice output unit ex208.
the text data and the transmit/receive circuit unit ex301 Accordingly, Voice data included in the moving picture file
performs digital-to-analog conversion processing and fre linked to a Web page, for instance, is reproduced.
quency transformation processing on it, the resultant is 0461) Note that the aforementioned system is not an
transmitted to the base station ex110 via the antenna eX201. exclusive example and therefore that at least either the
0455 When picture data is transmitted in data commu picture coding apparatus or the picture decoding apparatus
nication mode, the picture data shot by the camera unit of the above embodiments can be incorporated into a digital
ex203 is supplied to the picture coding unit ex312 via the broadcasting system as shown in FIG. 38, against the
camera interface unit ex303. When picture data is not to be backdrop that Satellite/terrestrial digital broadcasting has
transmitted, it is also possible to display Such picture data been a recent topic of conversation. To be more Specific, at
shot by the camera unit ex203 directly on the display unit a broadcasting station ex409, a bit stream of video infor
ex202 via the camera interface unit ex303 and the LCD mation is transmitted, by radio waves, to a satellite ex410 for
control unit ex302. communications or broadcasting. Upon receipt of it, the
broadcast satellite ex410 transmits radio waves for broad
0456. The picture coding unit ex312, which includes the casting, an antenna eX406 of a house equipped with Satellite
picture coding apparatus according to the present invention broadcasting reception facilities receives Such radio waves,
in its configuration, performs compression coding on the and an apparatus Such as a television (receiver) ex401 and a
picture data Supplied from the camera unit eX203 using the set top box (STP) ex407 decodes the bit stream and repro
coding method used by the picture coding apparatus pre duces the decoded data. The picture decoding apparatus as
Sented in the above-mentioned embodiments, So as to con shown in the above-mentioned embodiments can be imple
Vert it into coded picture data, and sends it out to the mented in the reproduction apparatus ex403 for reading and
multiplexing/demultiplexing unit ex308. At this time, the decoding the bit Stream recorded on a Storage medium ex402
cellular phone ex115 sends voices received by the voice that is a recording medium such as a CD and a DVD. In this
input unit ex205 while the shooting by the camera unit case, a reproduced Video Signal is displayed on a monitor
US 2004/0234143 A1 Nov. 25, 2004
29

eX404. It is also conceived that the picture decoding appa a Second reference picture Selection Step of Selecting, as
ratus is implemented in the set top box ex407 connected to a Second reference picture, any one of coded pictures
a cable ex405 for cable television or the antenna ex406 for located after the current picture in display order;
Satellite/terrestrial broadcasting So as to reproduce it on a a first reference picture Specification Step of Specifying, as
television monitor ex408. In this case, the picture decoding a first reference picture, any one of coded pictures
apparatus may be incorporated into the television, not in the based on a motion vector used for coding a predeter
Set top box. Or, a car ex412 with an antenna eX411 can mined block in the Second reference picture;
receive a signal from the satellite ex410, the base station
eX107 or the like, So as to reproduce a moving picture on a a predictive image generation Step of Specifying a block
display device Such as a car navigation System ex413 in the first reference picture and a block in the Second
mounted on the car ex412. reference picture based on the motion vector, and
0462 Furthermore, it is also possible to code a picture generating a predictive image from Said two blockS
Signal by the picture coding apparatus presented in the above through pixel interpolation;
embodiments and to record the resultant in a recording a coding Step of coding a difference between the predic
medium. Examples include a DVD recorder for recording a tive image and an image of a current block to be coded;
picture signal on a DVD disc ex421 and a recorder ex420 and
Such as a disc recorder for recording a picture Signal on a an output Step of outputting Specification information for
hard disk. Moreover, a picture Signal can also be recorded in Specifying the Second reference picture, together with a
an SD card ex422. If the recorder ex420 is equipped with the result of the coding performed in the coding Step.
picture decoding apparatus presented in the above embodi 2. The picture coding method according to claim 1,
ments, it is possible to reproduce a picture Signal recorded
on the DVD disc ex421 or in the SD card ex422, and display wherein in the output Step, the Specification information
it on the monitor ex408. that includes a Second reference index indicating the
0463 As the configuration of the car navigation system Second reference picture is outputted.
ex413, the configuration without the camera unit ex203 and 3. The picture coding method according to claim 1,
the camera interface unit ex303, out of the configuration wherein in the output Step, information indicating a
shown in FIG. 37, is conceivable. The same is applicable to method for changing Second reference indices is out
the computer ex111, the television (receiver) ex401 and the putted as the Specification information, Said Second
like. reference indices being assigned to the respective
0464 Concerning the terminals such as the cellular phone coded pictures in order to identify Said coded pictures.
eX114, a transmitting/receiving terminal having both an 4. A picture coding method for coding a current picture to
encoder and a decoder, as well as a transmitting terminal be coded on a block-by-block basis with reference to already
only with an encoder, and a receiving terminal only with a coded pictures, comprising:
decoder are possible as forms of implementation. a Second reference picture Selection Step of Selecting, as
0465. As stated above, it is possible to employ the picture a Second reference picture, a coded picture from among
coding method and the picture decoding method presented coded pictures that are located after the current picture
in the above embodiments into any one of the above in display order, Said coded picture being assigned a
Smallest Second reference index of all Second reference
described devices and Systems. Accordingly, it becomes indices assigned to the respective coded pictures in
possible to achieve an effect described in the aforementioned order to identify Said coded pictures,
embodiments.
a first reference picture Specification Step of Specifying, as
0466. From the invention thus described, it will be obvi a first reference picture, any one of coded pictures
ous that the embodiments of the invention may be varied in based on a motion vector used for coding a predeter
many ways. Such variations are not to be regarded as a mined block in the Second reference picture;
departure from the Spirit and Scope of the invention, and all
Such modifications as would be obvious to one skilled in the a predictive image generation Step of Specifying a block
art are intended for inclusion within the Scope of the in the first reference picture and a block in the Second
following claims. reference picture based on the motion vector, and
generating a predictive image from Said two blockS
Industrial Applicability through pixel interpolation; and
0467. The picture coding method and the picture decod a coding Step of coding a difference between the predic
ing method according to the present invention is Suited for tive image and an image of a current block to be coded.
use in a picture coding apparatus for coding a moving 5. The picture coding method according to claim 4,
picture and a picture decoding apparatus for decoding the
coded moving picture, as well as in a System equipped with wherein in the Second reference picture Selection Step, the
these apparatuses Such as a content Supply System for coded picture with the Smallest Second reference index
Supplying content like digital work and a digital broadcast is Specified according to assignment information indi
System. cating how the Second reference indices are assigned,
and Said Specified coded picture is Selected as the
Second reference picture.
1. A picture coding method for coding a current picture to 6. A picture coding method for coding a current picture to
be coded on a block-by-block basis with reference to already be coded on a block-by-block basis with reference to already
coded pictures, comprising: coded pictures, comprising:
US 2004/0234143 A1 Nov. 25, 2004
30

an assignment Step of assigning, to respective coded 9. A picture coding method for coding a current picture to
pictures, Second reference indices for identifying Said be coded on a block-by-block basis with reference to already
coded pictures in a manner in which a Second reference coded pictures, comprising:
indeX indicating 0 is assigned to any one of coded an assignment Step of assigning, to respective coded
pictures located after the current picture in display pictures, Second reference indices for identifying Said
order; coded pictures,
a re-assignment Step, in which it is possible to re-assign
the Second reference indices in a manner different from a Second reference picture Selection Step of Selecting, as
the manner used in the assignment Step, of assigning a Second reference picture, a coded picture which is
the Second reference indeX indicating 0 only to any one assigned a Second reference indeX indicating 0, from
of the coded pictures located after the current picture in among coded pictures located after the current picture
display order; in display order;
a Second reference picture Selection Step of Selecting, as a first reference picture Specification Step of Specifying, as
a Second reference picture, the coded picture assigned a first reference picture, any one of the coded pictures
the Second reference indeX indicating 0 from among the based on a motion vector used for coding a predeter
coded pictures located after the current picture in mined block in the Second reference picture;
display order; a predictive image generation Step of Specifying a block
a first reference picture Specification Step of Specifying, as in the first reference picture and a block in the Second
a first reference picture, any one of the coded pictures reference picture based on the motion vector, and
based on a motion vector used for coding a predeter generating a predictive image from Said two blockS
mined block in the Second reference picture; through pixel interpolation; and
a predictive image generation Step of Specifying a block a coding Step of coding a difference between the predic
in the first reference picture and a block in the Second tive image and an image of a current block to be coded,
reference picture based on the motion vector, and wherein in the assignment Step, the Second reference
generating a predictive image from Said two blockS indeX indicating 0 is prohibited from being assigned to
through pixel interpolation; and a coded picture that has been located before the current
a coding Step of coding a difference between the predic picture in display order.
tive image and an image of a current block to be coded. 10. The picture coding method according to claim 9,
7. The picture coding method according to claim 6, further comprising a deletion Step of deleting, from a
memory Storing motion vectors used for coding the coded
wherein in the re-assignment Step, it is prohibited to pictures, a motion vector used for coding the coded picture
change the Second reference index of the coded picture that has been located before the current picture in display
which is assigned the Second reference indeX indicating order.
0 in the assignment Step. 11. A picture coding method for coding a current picture
8. A picture coding method for coding a current picture to to be coded on a block-by-block basis with reference to
be coded on a block-by-block basis with reference to already already coded pictures, comprising:
coded pictures, comprising:
a Selection Step of Selecting, from among a plurality of
an assignment Step of assigning, to respective coded coded pictures Stored in a storage unit, a coded picture
pictures, first reference indices for identifying Said corresponding to a first reference indeX as a first
coded pictures, reference picture and a coded picture corresponding to
a re-assignment Step of re-assigning the first reference a Second reference indeX as a Second reference picture,
indices to predetermined coded pictures or Slices in a by use of Said first reference indeX and Said Second
manner different from the manner used in the assign reference indeX assigned to Said plurality of coded
ment Step; pictures,
a first reference picture Selection Step of Selecting, as a a predictive image generation Step of generating a pre
first reference picture, a coded picture assigned a first dictive image from a block in the first reference picture
reference indeX indicating a predetermined value, from and a block in the Second reference picture through
among coded pictures located before the current picture pixel interpolation; and
in display order; an output Step of coding a prediction error which is a
a predictive image generation Step of generating a pre difference between the current picture and the predic
dictive image through pixel interpolation based on a tive image, and outputting a coded picture Signal that
predetermined block in the first reference picture, and includes a signal obtained by coding Said prediction
error,
a coding Step of coding a difference between the predic
tive image and an image of a current block to be coded, wherein in the Selection Step, the coded picture which
Satisfies the following condition is Selected as the first
wherein in the re-assignment Step, the first reference indeX reference picture from among the coded pictures: a
indicating Said predetermined value is assigned prefer coded picture which is located before the current pic
entially to the coded picture which is referred to for a ture in display order and which is assigned a Smallest
long period of time. reference index, and
US 2004/0234143 A1 Nov. 25, 2004
31

in the predictive image generation step, (i) a Second 14. A picture decoding method for decoding a coded
motion vector of a block which refers to the first picture on a block-by-block basis with reference to already
reference picture is calculated, based on a first motion decoded pictures, comprising:
vector used to code Said block which is in the Second
reference picture and co-located with a current block to an obtainment Step of obtaining Specification information
be coded in the current picture, (ii) a third motion and a coded signal, the Specification information being
vector of the current block which refers to the first used for Specifying, as a Second reference picture, a
reference picture is calculated and a fourth motion decoded picture to be referred to for decoding a current
vector of the current block which refers to the second picture to be decoded and the coded signal being
reference picture is calculated, both based on Said related to Said current picture;
Second motion vector, and (iii) the predictive image is a Second reference picture Selection Step of Selecting the
generated through pixel interpolation from a block in Second reference picture from among decoded-pictures
the first reference picture referred to by the third motion that are located after the current picture in display order
vector and a block in the Second reference picture based on the Specification information;
referred to by the fourth motion vector. a first reference picture Specification Step of Specifying, as
12. A picture coding method for coding a current picture a first reference picture, any one of decoded pictures
to be coded on a block-by-block basis with reference to based on a motion vector used for decoding a prede
already coded pictures, comprising: termined block in the Second reference picture;
a Selection Step of Selecting, from among a plurality of a predictive image generation Step of Specifying a block
coded pictures Stored in a Storage unit, a coded picture in the first reference picture and a block in the Second
corresponding to a first reference indeX as a first reference picture based on the motion vector, and
reference picture and a coded picture corresponding to generating a predictive image from Said two blockS
a Second reference indeX as a Second reference picture, through pixel interpolation; and
by use of Said first reference indeX and Said Second
reference indeX assigned to Said plurality of coded a decoding Step of decoding a current block to be decoded
pictures, based on the predictive image and the coded signal.
a predictive image generation Step of generating a pre 15. The picture decoding method according to claim 14,
dictive image from a block in the first reference picture wherein in the obtainment step, the specification infor
and a block in the Second reference picture through mation that includes a Second reference indeX indicat
pixel interpolation; and ing the Second reference picture is obtained, and
an output Step of coding a prediction error which is a in the Second reference picture Selection Step, the Second
difference between the current picture and the predic reference picture indicated by Said Second reference
tive image, and outputting a coded picture Signal that indeX is Selected.
includes a signal obtained by coding Said prediction 16. The picture decoding method according to claim 14,
error,
wherein in the obtainment Step, the Specification infor
wherein in the Selection Step, the coded picture located mation that indicates a method for changing Second
before the current picture in display order is Selected as reference indices is obtained, Said Second reference
the first reference picture from among the coded pic indices being assigned to the respective decoded pic
tureS, tures in order to identify Said decoded pictures, and
in the predictive image generation step, (i) a Second in the Second reference picture Selection Step, the decoded
motion vector of a block which refers to the first picture indicated by Said method for changing the
reference picture is calculated, based on a first motion Second reference indices is Selected as the Second
vector used to code Said block which is in the Second reference picture.
reference picture and co-located with a current block to 17. A picture decoding method for decoding a coded
be coded in the current picture, (ii) a third motion picture on a block-by-block basis with reference to already
vector of the current block which refers to the first decoded pictures, comprising:
reference picture is calculated and a fourth motion an obtainment Step of obtaining a coded signal related to
vector of the current block which refers to the second
reference picture is calculated, both based on Said a current picture to be decoded;
Second motion vector, and (iii) the predictive image is a Second reference picture Selection Step of Selecting, as
generated through pixel interpolation from a block in a Second reference picture, a decoded picture from
the first reference picture referred to by the third motion among decoded pictures that are located after the
vector and a block in the Second reference picture current picture in display order, Said decoded picture
referred to by the fourth motion vector, and being assigned a Smallest Second reference index of all
in the output Step, the first reference index indicating the Second reference indices assigned to the respective
first reference picture Selected in the Selection Step is decoded pictures in order to identify Said decoded
incorporated into the coded picture Signal. pictures,
13. The picture coding method according to claim 12, a first reference picture Specification Step of Specifying, as
further comprising a Step of prohibiting information from a first reference picture, any one of the decoded pictures
being Stored into a memory, Said information indicating a based on a motion vector used for decoding a prede
coded picture referred to by the first motion vector. termined block in the Second reference picture;
US 2004/0234143 A1 Nov. 25, 2004
32

a predictive image generation Step of Specifying a block a decoding Step of decoding the current picture to be
in the first reference picture and a block in the Second decoded according to the predictive image and the
reference picture based on the motion vector, and prediction error; and
generating a predictive image from Said two blockS a storage Step of Storing, in the Storage unit, a decoded
through pixel interpolation; and picture with a possibility to be used as a reference
a decoding Step of decoding a current block to be decoded picture,
based on the predictive image and the coded signal. wherein in the Selection Step, the decoded picture which
18. The picture decoding method according to claim 17, Satisfies the following condition is Selected as the first
wherein in the Second reference picture Selection Step, the reference picture from among the decoded pictures: a
decoded picture with the Smallest Second reference decoded picture which is located before the current
indeX is Specified according to assignment information picture in display order and which is assigned a Small
indicating how the Second reference indices are est first reference index, and
assigned, and Said Specified decoded picture is Selected in the predictive image generation step, (i) a second
as the Second reference picture. motion vector of a block which refers to the first
19. A picture decoding method for decoding a coded reference picture is calculated, based on a first motion
picture on a block-by-block basis with reference to already Vector used to decode Said block which is in the Second
decoded pictures, comprising: reference picture and co-located with a current block to
an obtainment Step of obtaining a coded signal related to be decoded in the current picture, (ii) a third motion
a current picture to be decoded; vector of the current block which refers to the first
a Second reference picture Selection Step of Selecting, as reference picture is calculated and a fourth motion
vector of the current block which refers to the second
a Second reference picture, a decoded picture with a reference picture is calculated, both based on Said
Second reference index of 0, from among decoded Second motion vector, and (iii) the predictive image is
pictures which are located after the current picture in generated through pixel interpolation from a block in
display order and which are assigned Second reference the first reference picture referred to by the third motion
indices in order to identify Said respective decoded Vector and a block in the Second reference picture
pictures,
referred to by the fourth motion vector.
a first reference picture Specification Step of Specifying, as 21. A picture decoding method for decoding a coded
a first reference picture, any one of decoded pictures picture on a block-by-block basis with reference to already
based on a motion vector used for decoding a prede decoded pictures, comprising:
termined block in the Second reference picture; an obtainment Step of obtaining a coded picture Signal that
a predictive image generation Step of Specifying a block includes a coded picture Signal indicating a prediction
in the first reference picture and a block in the Second error,
reference picture based on the motion vector, and a Selection Step of Selecting, from among a plurality of
generating a predictive image from Said two blockS decoded pictures Stored in a storage unit, a decoded
through pixel interpolation; picture corresponding to a first reference indeX as a first
a decoding Step of decoding a current block to be decoded reference picture and a decoded picture corresponding
based on the predictive image and the coded signal, and to a Second reference indeX as a Second reference
a deletion Step of deleting, from a memory Storing motion picture, by use of Said first reference indeX and Said
vectors used for decoding the decoded pictures, a Second reference indeX assigned to Said plurality of
motion vector used for decoding a decoded picture that decoded pictures,
has been located before the current picture in display a predictive image generation Step of generating a pre
order. dictive image from a block in the first reference picture
20. A picture decoding method for decoding a coded and a block in the Second reference picture through
picture on a block-by-block basis with reference to already pixel interpolation;
decoded pictures, comprising: a decoding Step of decoding the current picture to be
an obtainment Step of obtaining a coded signal indicating decoded according to the predictive image and the
a prediction error; prediction error; and
a Selection Step of Selecting, from among a plurality of a storage Step of Storing, in the Storage unit, a decoded
decoded pictures Stored in a storage unit, a decoded picture with a possibility to be used as a reference
picture corresponding to a first reference indeX as a first picture, wherein in the Selection Step, the decoded
reference picture and a decoded picture corresponding picture indicated by the first reference indeX included in
to a Second reference indeX as a Second reference the coded picture Signal is Selected as the first reference
picture, by use of Said first reference indeX and Said picture from among the decoded pictures, and
Second reference indeX assigned to Said plurality of in the predictive image generation step, (i) a second
decoded pictures, motion vector of a block which refers to the first
a predictive image generation Step of generating a pre reference picture is calculated, based on a first motion
dictive image from a block in the first reference picture Vector used to decode Said block which is in the Second
and a block in the Second reference picture through reference picture and co-located with a current block to
pixel interpolation; be decoded in the current picture, (ii) a third motion
US 2004/0234143 A1 Nov. 25, 2004
33

vector of the current block which refers to the first a decoding unit operable to decode a current block to be
reference picture is calculated and a fourth motion decoded based on the predictive image and the coded
vector of the current block which refers to the second Signal.
reference picture is calculated, both based on Said 24. A program for causing a computer to execute a picture
Second motion vector, and (iii) the predictive image is coding method for coding a current picture to be coded on
generated through pixel interpolation from a block in a block-by-block basis with reference to already coded
the first reference picture referred to by the third motion pictures, comprising:
vector and a block in the Second reference picture
referred to by the fourth motion vector. a Second reference picture Selection Step of Selecting, as
22. A picture coding apparatus for coding a current picture a Second reference picture, any one of coded pictures
to be coded on a block-by-block basis with reference to located after the current picture in display order;
already coded pictures, comprising: a first reference picture Specification Step of Specifying, as
a Second reference picture Selection unit operable to a first reference picture, any one of coded pictures
Select, as a Second reference picture, any one of coded based on a motion vector used for coding a predeter
pictures located after the current picture in display mined block in the Second reference picture;
order; a predictive image generation Step of Specifying a block
first reference picture Specification unit operable to in the first reference picture and a block in the Second
Specify, as a first reference picture, any one of coded reference picture based on the motion vector, and
pictures based on a motion vector used for coding a generating a predictive image from Said two blockS
predetermined block in the Second reference picture; through pixel interpolation;
a predictive image generation unit operable to Specify a a coding Step of coding a difference between the predic
block in the first reference picture and a block in the tive image and an image of a current block to be coded;
Second reference picture based on the motion vector, and
and generate a predictive image from Said two blockS an output Step of outputting Specification information for
through pixel interpolation;
Specifying the Second reference picture, together with a
a coding unit operable to code a difference between the result of the coding performed in the coding Step.
predictive image and an image of a current block to be 25. A program for causing a computer to execute a picture
coded; and decoding method for decoding a coded picture on a block
an output unit operable to output Specification informa by-block basis with reference to already decoded pictures,
tion for Specifying the Second reference picture, comprising:
together with a result of the coding performed by the an obtainment Step of obtaining Specification information
coding unit. and a coded signal, the Specification information being
23. A picture decoding apparatus for decoding a coded used for Specifying, as a Second reference picture, a
picture on a block-by-block basis with reference to already decoded picture to be referred to for decoding a current
decoded pictures, comprising: picture to be decoded and the coded signal being
an obtainment unit operable to obtain Specification infor related to Said current picture;
mation and a coded Signal, the Specification informa a Second reference picture Selection Step of Selecting the
tion being used for Specifying, as a Second reference Second reference picture from among decoded pictures
picture, a decoded picture to be referred to for decoding that are located after the current picture in display order
a current picture to be decoded and the coded Signal based on the Specification information;
being related to Said current picture;
a Second reference picture Selection unit operable to Select a first reference picture Specification Step of Specifying, as
the Second reference picture from among decoded a first reference picture, any one of decoded pictures
pictures that are located after the current picture in based on a motion vector used for decoding a prede
display order based on the Specification information; termined block in the Second reference picture;
a first reference picture Specification unit operable to a predictive image generation Step of Specifying a block
Specify, as a first reference picture, any one of decoded in the first reference picture and a block in the Second
pictures based on a motion vector used for decoding a reference picture based on the motion vector, and
predetermined block in the Second reference picture; generating a predictive image from Said two blockS
through pixel interpolation; and
a predictive image generation unit operable to Specify a
block in the first reference picture and a block in the a decoding Step of decoding a current block to be decoded
Second reference picture based on the motion vector, based on the predictive image and the coded signal.
and generate a predictive image from Said two blockS
through pixel interpolation; and