Skip to content

Commit 66e5fce

Browse files
vpisarevvpisarev
authored
Merge pull request #27499 from vpisarev:image_io_with_metadata
Extend image I/O API with metadata support #27499 Covered with the PR: * AVIF encoder can write exif, xmp, icc * AVIF decoder can read exif * JPEG encoder can write exif * JPEG decoder can read exif * PNG encoder can write exif * PNG decoder can read exif This PR is a sort of preamble for #27488. I suggest to merge this one first to OpenCV 4.x, then promote this change to OpenCV 5.x and then provide extra API to read and write metadata in 5.x (or maybe 4.x) in a style similar to #27488. Maybe in that PR exif packing/unpacking should be done using a separate external API. That is, metadata reading and writing can/should be done in 2 steps: * [1] pack and then [2] embed exif into image at the encoding stage. * [1] extract and then [2] unpack exif at the decoding stage. ### Pull Request Readiness Checklist See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request - [x] I agree to contribute to the project under Apache 2 License. - [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV - [x] The PR is proposed to the proper branch - [x] There is a reference to the original bug report and related work - [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable Patch to opencv_extra has the same branch name. - [ ] The feature is well documented and sample code can be built with the project CMake
1 parent 677c4ee commit 66e5fce

File tree

11 files changed

+589
-14
lines changed

11 files changed

+589
-14
lines changed

‎modules/imgcodecs/include/opencv2/imgcodecs.hpp

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -251,6 +251,15 @@ enum ImwriteGIFCompressionFlags {
251251
IMWRITE_GIF_COLORTABLE_SIZE_256 = 8
252252
};
253253

254+
enum ImageMetadataType
255+
{
256+
IMAGE_METADATA_UNKNOWN = -1,
257+
IMAGE_METADATA_EXIF = 0,
258+
IMAGE_METADATA_XMP = 1,
259+
IMAGE_METADATA_ICCP = 2,
260+
IMAGE_METADATA_MAX = 2
261+
};
262+
254263
//! @} imgcodecs_flags
255264

256265
/** @brief Represents an animation with multiple frames.
@@ -360,6 +369,17 @@ The image passing through the img parameter can be pre-allocated. The memory is
360369
*/
361370
CV_EXPORTS_W void imread( const String& filename, OutputArray dst, int flags = IMREAD_COLOR_BGR );
362371

372+
/** @brief Reads an image from a file together with associated metadata.
373+
374+
The function imreadWithMetadata reads image from the specified file. It does the same thing as imread, but additionally reads metadata if the corresponding file contains any.
375+
@param filename Name of the file to be loaded.
376+
@param metadataTypes Output vector with types of metadata chucks returned in metadata, see ImageMetadataType.
377+
@param metadata Output vector of vectors or vector of matrices to store the retrieved metadata
378+
@param flags Flag that can take values of cv::ImreadModes
379+
*/
380+
CV_EXPORTS_W Mat imreadWithMetadata( const String& filename, CV_OUT std::vector<int>& metadataTypes,
381+
OutputArrayOfArrays metadata, int flags = IMREAD_ANYCOLOR);
382+
363383
/** @brief Loads a multi-page image from a file.
364384
365385
The function imreadmulti loads a multi-page image from the specified file into a vector of Mat objects.
@@ -508,6 +528,20 @@ It also demonstrates how to save multiple images in a TIFF file:
508528
CV_EXPORTS_W bool imwrite( const String& filename, InputArray img,
509529
const std::vector<int>& params = std::vector<int>());
510530

531+
/** @brief Saves an image to a specified file with metadata
532+
533+
The function imwriteWithMetadata saves the image to the specified file. It does the same thing as imwrite, but additionally writes metadata if the corresponding format supports it.
534+
@param filename Name of the file. As with imwrite, image format is determined by the file extension.
535+
@param img (Mat or vector of Mat) Image or Images to be saved.
536+
@param metadataTypes Vector with types of metadata chucks stored in metadata to write, see ImageMetadataType.
537+
@param metadata Vector of vectors or vector of matrices with chunks of metadata to store into the file
538+
@param params Format-specific parameters encoded as pairs (paramId_1, paramValue_1, paramId_2, paramValue_2, ... .) see cv::ImwriteFlags
539+
*/
540+
CV_EXPORTS_W bool imwriteWithMetadata( const String& filename, InputArray img,
541+
const std::vector<int>& metadataTypes,
542+
InputArrayOfArrays& metadata,
543+
const std::vector<int>& params = std::vector<int>());
544+
511545
//! @brief multi-image overload for bindings
512546
CV_WRAP static inline
513547
bool imwritemulti(const String& filename, InputArrayOfArrays img,
@@ -529,6 +563,22 @@ See cv::imread for the list of supported formats and flags description.
529563
*/
530564
CV_EXPORTS_W Mat imdecode( InputArray buf, int flags );
531565

566+
/** @brief Reads an image from a buffer in memory together with associated metadata.
567+
568+
The function imdecode reads an image from the specified buffer in the memory. If the buffer is too short or
569+
contains invalid data, the function returns an empty matrix ( Mat::data==NULL ).
570+
571+
See cv::imread for the list of supported formats and flags description.
572+
573+
@note In the case of color images, the decoded images will have the channels stored in **B G R** order.
574+
@param buf Input array or vector of bytes.
575+
@param metadataTypes Output vector with types of metadata chucks returned in metadata, see ImageMetadataType.
576+
@param metadata Output vector of vectors or vector of matrices to store the retrieved metadata
577+
@param flags The same flags as in cv::imread, see cv::ImreadModes.
578+
*/
579+
CV_EXPORTS_W Mat imdecodeWithMetadata( InputArray buf, CV_OUT std::vector<int>& metadataTypes,
580+
OutputArrayOfArrays metadata, int flags = IMREAD_ANYCOLOR );
581+
532582
/** @overload
533583
@param buf Input array or vector of bytes.
534584
@param flags The same flags as in cv::imread, see cv::ImreadModes.
@@ -567,6 +617,24 @@ CV_EXPORTS_W bool imencode( const String& ext, InputArray img,
567617
CV_OUT std::vector<uchar>& buf,
568618
const std::vector<int>& params = std::vector<int>());
569619

620+
/** @brief Encodes an image into a memory buffer.
621+
622+
The function imencode compresses the image and stores it in the memory buffer that is resized to fit the
623+
result. See cv::imwrite for the list of supported formats and flags description.
624+
625+
@param ext File extension that defines the output format. Must include a leading period.
626+
@param img Image to be compressed.
627+
@param metadataTypes Vector with types of metadata chucks stored in metadata to write, see ImageMetadataType.
628+
@param metadata Vector of vectors or vector of matrices with chunks of metadata to store into the file
629+
@param buf Output buffer resized to fit the compressed image.
630+
@param params Format-specific parameters. See cv::imwrite and cv::ImwriteFlags.
631+
*/
632+
CV_EXPORTS_W bool imencodeWithMetadata( const String& ext, InputArray img,
633+
const std::vector<int>& metadataTypes,
634+
InputArrayOfArrays metadata,
635+
CV_OUT std::vector<uchar>& buf,
636+
const std::vector<int>& params = std::vector<int>());
637+
570638
/** @brief Encodes array of images into a memory buffer.
571639
572640
The function is analog to cv::imencode for in-memory multi-page image compression.

‎modules/imgcodecs/src/exif.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,10 @@ ExifEntry_t ExifReader::getTag(const ExifTagName tag) const
9494
return entry;
9595
}
9696

97+
const std::vector<unsigned char>& ExifReader::getData() const
98+
{
99+
return m_data;
100+
}
97101

98102
/**
99103
* @brief Parsing the exif data buffer and prepare (internal) exif directory

‎modules/imgcodecs/src/exif.hpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -175,6 +175,10 @@ class ExifReader
175175
*/
176176
ExifEntry_t getTag( const ExifTagName tag ) const;
177177

178+
/**
179+
* @brief Get the whole exif buffer
180+
*/
181+
const std::vector<unsigned char>& getData() const;
178182

179183
private:
180184
std::vector<unsigned char> m_data;

‎modules/imgcodecs/src/grfmt_avif.cpp

Lines changed: 20 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -68,8 +68,8 @@ avifResult CopyToMat(const avifImage *image, int channels, bool useRGB , Mat *ma
6868
return avifImageYUVToRGB(image, &rgba);
6969
}
7070

71-
AvifImageUniquePtr ConvertToAvif(const cv::Mat &img, bool lossless,
72-
int bit_depth) {
71+
AvifImageUniquePtr ConvertToAvif(const cv::Mat &img, bool lossless, int bit_depth,
72+
const std::vector<std::vector<uchar> >& metadata) {
7373
CV_Assert(img.depth() == CV_8U || img.depth() == CV_16U);
7474

7575
const int width = img.cols;
@@ -112,6 +112,18 @@ AvifImageUniquePtr ConvertToAvif(const cv::Mat &img, bool lossless,
112112
result->yuvRange = AVIF_RANGE_FULL;
113113
}
114114

115+
if (!metadata.empty()) {
116+
const std::vector<uchar>& metadata_exif = metadata[IMAGE_METADATA_EXIF];
117+
const std::vector<uchar>& metadata_xmp = metadata[IMAGE_METADATA_XMP];
118+
const std::vector<uchar>& metadata_iccp = metadata[IMAGE_METADATA_ICCP];
119+
if (!metadata_exif.empty())
120+
avifImageSetMetadataExif(result, (const uint8_t*)metadata_exif.data(), metadata_exif.size());
121+
if (!metadata_exif.empty())
122+
avifImageSetMetadataXMP(result, (const uint8_t*)metadata_xmp.data(), metadata_xmp.size());
123+
if (!metadata_iccp.empty())
124+
avifImageSetProfileICC(result, (const uint8_t*)metadata_iccp.data(), metadata_iccp.size());
125+
}
126+
115127
avifRGBImage rgba;
116128
avifRGBImageSetDefaults(&rgba, result);
117129
if (img.channels() == 3) {
@@ -120,7 +132,7 @@ AvifImageUniquePtr ConvertToAvif(const cv::Mat &img, bool lossless,
120132
CV_Assert(img.channels() == 4);
121133
rgba.format = AVIF_RGB_FORMAT_BGRA;
122134
}
123-
rgba.rowBytes = img.step[0];
135+
rgba.rowBytes = (uint32_t)img.step[0];
124136
rgba.depth = bit_depth;
125137
rgba.pixels =
126138
const_cast<uint8_t *>(reinterpret_cast<const uint8_t *>(img.data));
@@ -287,6 +299,10 @@ bool AvifDecoder::nextPage() {
287299
AvifEncoder::AvifEncoder() {
288300
m_description = "AVIF files (*.avif)";
289301
m_buf_supported = true;
302+
m_support_metadata.assign((size_t)IMAGE_METADATA_MAX + 1, false);
303+
m_support_metadata[(size_t)IMAGE_METADATA_EXIF] = true;
304+
m_support_metadata[(size_t)IMAGE_METADATA_XMP] = true;
305+
m_support_metadata[(size_t)IMAGE_METADATA_ICCP] = true;
290306
encoder_ = avifEncoderCreate();
291307
}
292308

@@ -349,7 +365,7 @@ bool AvifEncoder::writeanimation(const Animation& animation,
349365
img.channels() == 1 || img.channels() == 3 || img.channels() == 4,
350366
"AVIF only supports 1, 3, 4 channels");
351367

352-
images.emplace_back(ConvertToAvif(img, do_lossless, bit_depth));
368+
images.emplace_back(ConvertToAvif(img, do_lossless, bit_depth, m_metadata));
353369
}
354370

355371
for (size_t i = 0; i < images.size(); i++)

‎modules/imgcodecs/src/grfmt_base.cpp

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,11 +58,30 @@ BaseImageDecoder::BaseImageDecoder()
5858
m_frame_count = 1;
5959
}
6060

61+
bool BaseImageDecoder::haveMetadata(ImageMetadataType type) const
62+
{
63+
if (type == IMAGE_METADATA_EXIF)
64+
return !m_exif.getData().empty();
65+
return false;
66+
}
67+
68+
Mat BaseImageDecoder::getMetadata(ImageMetadataType type) const
69+
{
70+
if (type == IMAGE_METADATA_EXIF) {
71+
const std::vector<unsigned char>& exif = m_exif.getData();
72+
if (!exif.empty()) {
73+
Mat exifmat(1, (int)exif.size(), CV_8U, (void*)exif.data());
74+
return exifmat;
75+
}
76+
}
77+
return Mat();
78+
}
6179

6280
ExifEntry_t BaseImageDecoder::getExifTag(const ExifTagName tag) const
6381
{
6482
return m_exif.getTag(tag);
6583
}
84+
6685
bool BaseImageDecoder::setSource( const String& filename )
6786
{
6887
m_filename = filename;
@@ -140,6 +159,23 @@ bool BaseImageEncoder::setDestination( std::vector<uchar>& buf )
140159
return true;
141160
}
142161

162+
bool BaseImageEncoder::addMetadata(ImageMetadataType type, const Mat& metadata)
163+
{
164+
CV_Assert_N(type >= IMAGE_METADATA_EXIF, type <= IMAGE_METADATA_MAX);
165+
if (metadata.empty())
166+
return true;
167+
size_t itype = (size_t)type;
168+
if (itype >= m_support_metadata.size() || !m_support_metadata[itype])
169+
return false;
170+
if (m_metadata.empty())
171+
m_metadata.resize((size_t)IMAGE_METADATA_MAX+1);
172+
CV_Assert(metadata.elemSize() == 1);
173+
CV_Assert(metadata.isContinuous());
174+
const unsigned char* data = metadata.ptr<unsigned char>();
175+
m_metadata[itype].assign(data, data + metadata.total());
176+
return true;
177+
}
178+
143179
bool BaseImageEncoder::write(const Mat &img, const std::vector<int> &params) {
144180
std::vector<Mat> img_vec(1, img);
145181
return writemulti(img_vec, params);

‎modules/imgcodecs/src/grfmt_base.hpp

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,6 +69,20 @@ class BaseImageDecoder {
6969
*/
7070
virtual int type() const { return m_type; }
7171

72+
/**
73+
* @brief Checks whether file contains metadata of the certain type.
74+
* @param type The type of metadata to look for
75+
*/
76+
virtual bool haveMetadata(ImageMetadataType type) const;
77+
78+
/**
79+
* @brief Retrieves metadata (if any) of the certain kind.
80+
* If there is no such metadata, the method returns empty array.
81+
*
82+
* @param type The type of metadata to look for
83+
*/
84+
virtual Mat getMetadata(ImageMetadataType type) const;
85+
7286
/**
7387
* @brief Fetch a specific EXIF tag from the image's metadata.
7488
* @param tag The EXIF tag to retrieve.
@@ -205,6 +219,13 @@ class BaseImageEncoder {
205219
*/
206220
virtual bool setDestination(std::vector<uchar>& buf);
207221

222+
/**
223+
* @brief Sets the metadata to write together with the image data
224+
* @param type The type of metadata to add
225+
* @param metadata The packed metadata (Exif, XMP, ...)
226+
*/
227+
virtual bool addMetadata(ImageMetadataType type, const Mat& metadata);
228+
208229
/**
209230
* @brief Encode and write the image data.
210231
* @param img The Mat object containing the image data to be encoded.
@@ -243,6 +264,8 @@ class BaseImageEncoder {
243264
virtual void throwOnError() const;
244265

245266
protected:
267+
std::vector<std::vector<unsigned char> > m_metadata; // see IMAGE_METADATA_...
268+
std::vector<bool> m_support_metadata;
246269
String m_description; ///< Description of the encoder (e.g., format name, capabilities).
247270
String m_filename; ///< Destination file name for encoded data.
248271
std::vector<uchar>* m_buf; ///< Pointer to the buffer for encoded data if using memory-based destination.

‎modules/imgcodecs/src/grfmt_jpeg.cpp

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -600,6 +600,8 @@ JpegEncoder::JpegEncoder()
600600
{
601601
m_description = "JPEG files (*.jpeg;*.jpg;*.jpe)";
602602
m_buf_supported = true;
603+
m_support_metadata.assign((size_t)IMAGE_METADATA_MAX + 1, false);
604+
m_support_metadata[(size_t)IMAGE_METADATA_EXIF] = true;
603605
}
604606

605607

@@ -815,6 +817,22 @@ bool JpegEncoder::write( const Mat& img, const std::vector<int>& params )
815817

816818
jpeg_start_compress( &cinfo, TRUE );
817819

820+
if (!m_metadata.empty()) {
821+
const std::vector<uchar>& metadata_exif = m_metadata[IMAGE_METADATA_EXIF];
822+
size_t exif_size = metadata_exif.size();
823+
if (exif_size > 0u) {
824+
const char app1_exif_prefix[] = {'E', 'x', 'i', 'f', '\0', '\0'};
825+
size_t app1_exif_prefix_size = sizeof(app1_exif_prefix);
826+
size_t data_size = exif_size + app1_exif_prefix_size;
827+
828+
std::vector<uchar> metadata_app1(data_size);
829+
uchar* data = metadata_app1.data();
830+
memcpy(data, app1_exif_prefix, app1_exif_prefix_size);
831+
memcpy(data + app1_exif_prefix_size, metadata_exif.data(), exif_size);
832+
jpeg_write_marker(&cinfo, JPEG_APP0 + 1, data, (unsigned)data_size);
833+
}
834+
}
835+
818836
if( doDirectWrite )
819837
{
820838
for( int y = 0; y < height; y++ )

‎modules/imgcodecs/src/grfmt_png.cpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -858,6 +858,8 @@ PngEncoder::PngEncoder()
858858
{
859859
m_description = "Portable Network Graphics files (*.png;*.apng)";
860860
m_buf_supported = true;
861+
m_support_metadata.assign((size_t)IMAGE_METADATA_MAX+1, false);
862+
m_support_metadata[IMAGE_METADATA_EXIF] = true;
861863
op_zstream1.zalloc = NULL;
862864
op_zstream2.zalloc = NULL;
863865
next_seq_num = 0;
@@ -989,6 +991,16 @@ bool PngEncoder::write( const Mat& img, const std::vector<int>& params )
989991
for( y = 0; y < height; y++ )
990992
buffer[y] = img.data + y*img.step;
991993

994+
if (!m_metadata.empty()) {
995+
std::vector<uchar>& exif = m_metadata[IMAGE_METADATA_EXIF];
996+
if (!exif.empty()) {
997+
writeChunk(f, "eXIf", exif.data(), (uint32_t)exif.size());
998+
}
999+
// [TODO] add xmp and icc. They need special handling,
1000+
// see https://dev.exiv2.org/projects/exiv2/wiki/The_Metadata_in_PNG_files and
1001+
// https://www.libpng.org/pub/png/spec/1.2/PNG-Chunks.html.
1002+
}
1003+
9921004
png_write_image( png_ptr, buffer.data() );
9931005
png_write_end( png_ptr, info_ptr );
9941006

0 commit comments

Comments
 (0)