Skip to content

Commit c27bb30

Browse files
paulkocialkowskimchehab
paulkocialkowski
authored and
mchehab
committed
media: v4l: Add definitions for MPEG-2 slice format and metadata
Stateless video decoding engines require both the MPEG-2 slices and associated metadata from the video stream in order to decode frames. This introduces definitions for a new pixel format, describing buffers with MPEG-2 slice data, as well as control structure sfor passing the frame metadata to drivers. This is based on work from both Florent Revest and Hugues Fruchet. Signed-off-by: Paul Kocialkowski <paul.kocialkowski@bootlin.com> Signed-off-by: Maxime Ripard <maxime.ripard@bootlin.com> Signed-off-by: Hans Verkuil <hans.verkuil@cisco.com> Signed-off-by: Mauro Carvalho Chehab <mchehab+samsung@kernel.org>
1 parent 515c5a7 commit c27bb30

File tree

9 files changed

+351
-9
lines changed

9 files changed

+351
-9
lines changed

‎Documentation/media/uapi/v4l/extended-controls.rst

Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1497,6 +1497,182 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
14971497

14981498

14991499

1500+
.. _v4l2-mpeg-mpeg2:
1501+
1502+
``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS (struct)``
1503+
Specifies the slice parameters (as extracted from the bitstream) for the
1504+
associated MPEG-2 slice data. This includes the necessary parameters for
1505+
configuring a stateless hardware decoding pipeline for MPEG-2.
1506+
The bitstream parameters are defined according to :ref:`mpeg2part2`.
1507+
1508+
.. c:type:: v4l2_ctrl_mpeg2_slice_params
1509+
1510+
.. cssclass:: longtable
1511+
1512+
.. flat-table:: struct v4l2_ctrl_mpeg2_slice_params
1513+
:header-rows: 0
1514+
:stub-columns: 0
1515+
:widths: 1 1 2
1516+
1517+
* - __u32
1518+
- ``bit_size``
1519+
- Size (in bits) of the current slice data.
1520+
* - __u32
1521+
- ``data_bit_offset``
1522+
- Offset (in bits) to the video data in the current slice data.
1523+
* - struct :c:type:`v4l2_mpeg2_sequence`
1524+
- ``sequence``
1525+
- Structure with MPEG-2 sequence metadata, merging relevant fields from
1526+
the sequence header and sequence extension parts of the bitstream.
1527+
* - struct :c:type:`v4l2_mpeg2_picture`
1528+
- ``picture``
1529+
- Structure with MPEG-2 picture metadata, merging relevant fields from
1530+
the picture header and picture coding extension parts of the bitstream.
1531+
* - __u8
1532+
- ``quantiser_scale_code``
1533+
- Code used to determine the quantization scale to use for the IDCT.
1534+
* - __u8
1535+
- ``backward_ref_index``
1536+
- Index for the V4L2 buffer to use as backward reference, used with
1537+
B-coded and P-coded frames.
1538+
* - __u8
1539+
- ``forward_ref_index``
1540+
- Index for the V4L2 buffer to use as forward reference, used with
1541+
B-coded frames.
1542+
1543+
.. c:type:: v4l2_mpeg2_sequence
1544+
1545+
.. cssclass:: longtable
1546+
1547+
.. flat-table:: struct v4l2_mpeg2_sequence
1548+
:header-rows: 0
1549+
:stub-columns: 0
1550+
:widths: 1 1 2
1551+
1552+
* - __u16
1553+
- ``horizontal_size``
1554+
- The width of the displayable part of the frame's luminance component.
1555+
* - __u16
1556+
- ``vertical_size``
1557+
- The height of the displayable part of the frame's luminance component.
1558+
* - __u32
1559+
- ``vbv_buffer_size``
1560+
- Used to calculate the required size of the video buffering verifier,
1561+
defined (in bits) as: 16 * 1024 * vbv_buffer_size.
1562+
* - __u8
1563+
- ``profile_and_level_indication``
1564+
- The current profile and level indication as extracted from the
1565+
bitstream.
1566+
* - __u8
1567+
- ``progressive_sequence``
1568+
- Indication that all the frames for the sequence are progressive instead
1569+
of interlaced.
1570+
* - __u8
1571+
- ``chroma_format``
1572+
- The chrominance sub-sampling format (1: 4:2:0, 2: 4:2:2, 3: 4:4:4).
1573+
1574+
.. c:type:: v4l2_mpeg2_picture
1575+
1576+
.. cssclass:: longtable
1577+
1578+
.. flat-table:: struct v4l2_mpeg2_picture
1579+
:header-rows: 0
1580+
:stub-columns: 0
1581+
:widths: 1 1 2
1582+
1583+
* - __u8
1584+
- ``picture_coding_type``
1585+
- Picture coding type for the frame covered by the current slice
1586+
(V4L2_MPEG2_PICTURE_CODING_TYPE_I, V4L2_MPEG2_PICTURE_CODING_TYPE_P or
1587+
V4L2_MPEG2_PICTURE_CODING_TYPE_B).
1588+
* - __u8
1589+
- ``f_code[2][2]``
1590+
- Motion vector codes.
1591+
* - __u8
1592+
- ``intra_dc_precision``
1593+
- Precision of Discrete Cosine transform (0: 8 bits precision,
1594+
1: 9 bits precision, 2: 10 bits precision, 3: 11 bits precision).
1595+
* - __u8
1596+
- ``picture_structure``
1597+
- Picture structure (1: interlaced top field, 2: interlaced bottom field,
1598+
3: progressive frame).
1599+
* - __u8
1600+
- ``top_field_first``
1601+
- If set to 1 and interlaced stream, top field is output first.
1602+
* - __u8
1603+
- ``frame_pred_frame_dct``
1604+
- If set to 1, only frame-DCT and frame prediction are used.
1605+
* - __u8
1606+
- ``concealment_motion_vectors``
1607+
- If set to 1, motion vectors are coded for intra macroblocks.
1608+
* - __u8
1609+
- ``q_scale_type``
1610+
- This flag affects the inverse quantization process.
1611+
* - __u8
1612+
- ``intra_vlc_format``
1613+
- This flag affects the decoding of transform coefficient data.
1614+
* - __u8
1615+
- ``alternate_scan``
1616+
- This flag affects the decoding of transform coefficient data.
1617+
* - __u8
1618+
- ``repeat_first_field``
1619+
- This flag affects the decoding process of progressive frames.
1620+
* - __u8
1621+
- ``progressive_frame``
1622+
- Indicates whether the current frame is progressive.
1623+
1624+
``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION (struct)``
1625+
Specifies quantization matrices (as extracted from the bitstream) for the
1626+
associated MPEG-2 slice data.
1627+
1628+
.. c:type:: v4l2_ctrl_mpeg2_quantization
1629+
1630+
.. cssclass:: longtable
1631+
1632+
.. flat-table:: struct v4l2_ctrl_mpeg2_quantization
1633+
:header-rows: 0
1634+
:stub-columns: 0
1635+
:widths: 1 1 2
1636+
1637+
* - __u8
1638+
- ``load_intra_quantiser_matrix``
1639+
- One bit to indicate whether to load the ``intra_quantiser_matrix`` data.
1640+
* - __u8
1641+
- ``load_non_intra_quantiser_matrix``
1642+
- One bit to indicate whether to load the ``non_intra_quantiser_matrix``
1643+
data.
1644+
* - __u8
1645+
- ``load_chroma_intra_quantiser_matrix``
1646+
- One bit to indicate whether to load the
1647+
``chroma_intra_quantiser_matrix`` data, only relevant for non-4:2:0 YUV
1648+
formats.
1649+
* - __u8
1650+
- ``load_chroma_non_intra_quantiser_matrix``
1651+
- One bit to indicate whether to load the
1652+
``chroma_non_intra_quantiser_matrix`` data, only relevant for non-4:2:0
1653+
YUV formats.
1654+
* - __u8
1655+
- ``intra_quantiser_matrix[64]``
1656+
- The quantization matrix coefficients for intra-coded frames, in zigzag
1657+
scanning order. It is relevant for both luma and chroma components,
1658+
although it can be superseded by the chroma-specific matrix for
1659+
non-4:2:0 YUV formats.
1660+
* - __u8
1661+
- ``non_intra_quantiser_matrix[64]``
1662+
- The quantization matrix coefficients for non-intra-coded frames, in
1663+
zigzag scanning order. It is relevant for both luma and chroma
1664+
components, although it can be superseded by the chroma-specific matrix
1665+
for non-4:2:0 YUV formats.
1666+
* - __u8
1667+
- ``chroma_intra_quantiser_matrix[64]``
1668+
- The quantization matrix coefficients for the chominance component of
1669+
intra-coded frames, in zigzag scanning order. Only relevant for
1670+
non-4:2:0 YUV formats.
1671+
* - __u8
1672+
- ``chroma_non_intra_quantiser_matrix[64]``
1673+
- The quantization matrix coefficients for the chrominance component of
1674+
non-intra-coded frames, in zigzag scanning order. Only relevant for
1675+
non-4:2:0 YUV formats.
15001676

15011677
MFC 5.1 MPEG Controls
15021678
---------------------

‎Documentation/media/uapi/v4l/pixfmt-compressed.rst

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,22 @@ Compressed Formats
6060
- ``V4L2_PIX_FMT_MPEG2``
6161
- 'MPG2'
6262
- MPEG2 video elementary stream.
63+
* .. _V4L2-PIX-FMT-MPEG2-SLICE:
64+
65+
- ``V4L2_PIX_FMT_MPEG2_SLICE``
66+
- 'MG2S'
67+
- MPEG-2 parsed slice data, as extracted from the MPEG-2 bitstream.
68+
This format is adapted for stateless video decoders that implement a
69+
MPEG-2 pipeline (using the :ref:`codec` and :ref:`media-request-api`).
70+
Metadata associated with the frame to decode is required to be passed
71+
through the ``V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS`` control and
72+
quantization matrices can optionally be specified through the
73+
``V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION`` control.
74+
See the :ref:`associated Codec Control IDs <v4l2-mpeg-mpeg2>`.
75+
Exactly one output and one capture buffer must be provided for use with
76+
this pixel format. The output buffer must contain the appropriate number
77+
of macroblocks to decode a full corresponding frame to the matching
78+
capture buffer.
6379
* .. _V4L2-PIX-FMT-MPEG4:
6480

6581
- ``V4L2_PIX_FMT_MPEG4``

‎Documentation/media/uapi/v4l/vidioc-queryctrl.rst

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -424,8 +424,18 @@ See also the examples in :ref:`control`.
424424
- any
425425
- An unsigned 32-bit valued control ranging from minimum to maximum
426426
inclusive. The step value indicates the increment between values.
427-
428-
427+
* - ``V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS``
428+
- n/a
429+
- n/a
430+
- n/a
431+
- A struct :c:type:`v4l2_ctrl_mpeg2_slice_params`, containing MPEG-2
432+
slice parameters for stateless video decoders.
433+
* - ``V4L2_CTRL_TYPE_MPEG2_QUANTIZATION``
434+
- n/a
435+
- n/a
436+
- n/a
437+
- A struct :c:type:`v4l2_ctrl_mpeg2_quantization`, containing MPEG-2
438+
quantization matrices for stateless video decoders.
429439

430440
.. tabularcolumns:: |p{6.6cm}|p{2.2cm}|p{8.7cm}|
431441

‎Documentation/media/videodev2.h.rst.exceptions

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,8 @@ replace symbol V4L2_CTRL_TYPE_STRING :c:type:`v4l2_ctrl_type`
129129
replace symbol V4L2_CTRL_TYPE_U16 :c:type:`v4l2_ctrl_type`
130130
replace symbol V4L2_CTRL_TYPE_U32 :c:type:`v4l2_ctrl_type`
131131
replace symbol V4L2_CTRL_TYPE_U8 :c:type:`v4l2_ctrl_type`
132+
replace symbol V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS :c:type:`v4l2_ctrl_type`
133+
replace symbol V4L2_CTRL_TYPE_MPEG2_QUANTIZATION :c:type:`v4l2_ctrl_type`
132134

133135
# V4L2 capability defines
134136
replace define V4L2_CAP_VIDEO_CAPTURE device-capabilities

‎drivers/media/v4l2-core/v4l2-ctrls.c

Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -844,6 +844,8 @@ const char *v4l2_ctrl_get_name(u32 id)
844844
case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: return "Vertical MV Search Range";
845845
case V4L2_CID_MPEG_VIDEO_REPEAT_SEQ_HEADER: return "Repeat Sequence Header";
846846
case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: return "Force Key Frame";
847+
case V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS: return "MPEG-2 Slice Parameters";
848+
case V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION: return "MPEG-2 Quantization Matrices";
847849

848850
/* VPX controls */
849851
case V4L2_CID_MPEG_VIDEO_VPX_NUM_PARTITIONS: return "VPX Number of Partitions";
@@ -1292,6 +1294,12 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type,
12921294
case V4L2_CID_RDS_TX_ALT_FREQS:
12931295
*type = V4L2_CTRL_TYPE_U32;
12941296
break;
1297+
case V4L2_CID_MPEG_VIDEO_MPEG2_SLICE_PARAMS:
1298+
*type = V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS;
1299+
break;
1300+
case V4L2_CID_MPEG_VIDEO_MPEG2_QUANTIZATION:
1301+
*type = V4L2_CTRL_TYPE_MPEG2_QUANTIZATION;
1302+
break;
12951303
default:
12961304
*type = V4L2_CTRL_TYPE_INTEGER;
12971305
break;
@@ -1550,6 +1558,7 @@ static void std_log(const struct v4l2_ctrl *ctrl)
15501558
static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
15511559
union v4l2_ctrl_ptr ptr)
15521560
{
1561+
struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params;
15531562
size_t len;
15541563
u64 offset;
15551564
s64 val;
@@ -1612,6 +1621,54 @@ static int std_validate(const struct v4l2_ctrl *ctrl, u32 idx,
16121621
return -ERANGE;
16131622
return 0;
16141623

1624+
case V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS:
1625+
p_mpeg2_slice_params = ptr.p;
1626+
1627+
switch (p_mpeg2_slice_params->sequence.chroma_format) {
1628+
case 1: /* 4:2:0 */
1629+
case 2: /* 4:2:2 */
1630+
case 3: /* 4:4:4 */
1631+
break;
1632+
default:
1633+
return -EINVAL;
1634+
}
1635+
1636+
switch (p_mpeg2_slice_params->picture.intra_dc_precision) {
1637+
case 0: /* 8 bits */
1638+
case 1: /* 9 bits */
1639+
case 11: /* 11 bits */
1640+
break;
1641+
default:
1642+
return -EINVAL;
1643+
}
1644+
1645+
switch (p_mpeg2_slice_params->picture.picture_structure) {
1646+
case 1: /* interlaced top field */
1647+
case 2: /* interlaced bottom field */
1648+
case 3: /* progressive */
1649+
break;
1650+
default:
1651+
return -EINVAL;
1652+
}
1653+
1654+
switch (p_mpeg2_slice_params->picture.picture_coding_type) {
1655+
case V4L2_MPEG2_PICTURE_CODING_TYPE_I:
1656+
case V4L2_MPEG2_PICTURE_CODING_TYPE_P:
1657+
case V4L2_MPEG2_PICTURE_CODING_TYPE_B:
1658+
break;
1659+
default:
1660+
return -EINVAL;
1661+
}
1662+
1663+
if (p_mpeg2_slice_params->backward_ref_index >= VIDEO_MAX_FRAME ||
1664+
p_mpeg2_slice_params->forward_ref_index >= VIDEO_MAX_FRAME)
1665+
return -EINVAL;
1666+
1667+
return 0;
1668+
1669+
case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION:
1670+
return 0;
1671+
16151672
default:
16161673
return -EINVAL;
16171674
}
@@ -2186,6 +2243,12 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl,
21862243
case V4L2_CTRL_TYPE_U32:
21872244
elem_size = sizeof(u32);
21882245
break;
2246+
case V4L2_CTRL_TYPE_MPEG2_SLICE_PARAMS:
2247+
elem_size = sizeof(struct v4l2_ctrl_mpeg2_slice_params);
2248+
break;
2249+
case V4L2_CTRL_TYPE_MPEG2_QUANTIZATION:
2250+
elem_size = sizeof(struct v4l2_ctrl_mpeg2_quantization);
2251+
break;
21892252
default:
21902253
if (type < V4L2_CTRL_COMPOUND_TYPES)
21912254
elem_size = sizeof(s32);

‎drivers/media/v4l2-core/v4l2-ioctl.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1309,6 +1309,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
13091309
case V4L2_PIX_FMT_H263: descr = "H.263"; break;
13101310
case V4L2_PIX_FMT_MPEG1: descr = "MPEG-1 ES"; break;
13111311
case V4L2_PIX_FMT_MPEG2: descr = "MPEG-2 ES"; break;
1312+
case V4L2_PIX_FMT_MPEG2_SLICE: descr = "MPEG-2 Parsed Slice Data"; break;
13121313
case V4L2_PIX_FMT_MPEG4: descr = "MPEG-4 part 2 ES"; break;
13131314
case V4L2_PIX_FMT_XVID: descr = "Xvid"; break;
13141315
case V4L2_PIX_FMT_VC1_ANNEX_G: descr = "VC-1 (SMPTE 412M Annex G)"; break;

‎include/media/v4l2-ctrls.h

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,13 +35,15 @@ struct poll_table_struct;
3535

3636
/**
3737
* union v4l2_ctrl_ptr - A pointer to a control value.
38-
* @p_s32: Pointer to a 32-bit signed value.
39-
* @p_s64: Pointer to a 64-bit signed value.
40-
* @p_u8: Pointer to a 8-bit unsigned value.
41-
* @p_u16: Pointer to a 16-bit unsigned value.
42-
* @p_u32: Pointer to a 32-bit unsigned value.
43-
* @p_char: Pointer to a string.
44-
* @p: Pointer to a compound value.
38+
* @p_s32: Pointer to a 32-bit signed value.
39+
* @p_s64: Pointer to a 64-bit signed value.
40+
* @p_u8: Pointer to a 8-bit unsigned value.
41+
* @p_u16: Pointer to a 16-bit unsigned value.
42+
* @p_u32: Pointer to a 32-bit unsigned value.
43+
* @p_char: Pointer to a string.
44+
* @p_mpeg2_slice_params: Pointer to a MPEG2 slice parameters structure.
45+
* @p_mpeg2_quantization: Pointer to a MPEG2 quantization data structure.
46+
* @p: Pointer to a compound value.
4547
*/
4648
union v4l2_ctrl_ptr {
4749
s32 *p_s32;
@@ -50,6 +52,8 @@ union v4l2_ctrl_ptr {
5052
u16 *p_u16;
5153
u32 *p_u32;
5254
char *p_char;
55+
struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params;
56+
struct v4l2_ctrl_mpeg2_quantization *p_mpeg2_quantization;
5357
void *p;
5458
};
5559

0 commit comments

Comments
 (0)