Skip to content

bpo-45639: Add webp and avif image formats to mimetypes #29259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
May 3, 2022

Conversation

kixorz
Copy link
Contributor

@kixorz kixorz commented Oct 28, 2021

Modern image types webp and avif are not recognized by the mimetypes module.

Problem: Many tools are written in Python and running on macOS. Good example is the AWS CLI. Running commands like "s3 sync" will save files with .webp and .avif extensions with incorrect "binary/octet-stream" Content-Type to S3. This creates additional problems with serving these resources over HTTP.

The webp and avif image types are supported by most browsers:
https://caniuse.com/#feat=webp
https://caniuse.com/#feat=avif

While webp is fully supported and largely used, it is not officially registered with IANA.

Avif is currently less popular, but it is fully registered with IANA.
https://www.iana.org/assignments/media-types/media-types.xhtml

Please consider the attached GitHub PR as a fix to these MIME Content-Type issues.

https://bugs.python.org/issue45639

#89802

@kixorz kixorz requested a review from a team as a code owner October 28, 2021 02:35
@the-knights-who-say-ni
Copy link

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@kixorz

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@kixorz
Copy link
Contributor Author

kixorz commented Oct 28, 2021

I actually signed the CLA.

@merwok
Copy link
Member

merwok commented Oct 30, 2021

wepb is already added in #23034 (and with proper alpha sorting in the list)

@kixorz
Copy link
Contributor Author

kixorz commented Oct 31, 2021

I saw the other PR, but it's not merged yet. The addition in the commit in this PR is sorted by mime type as requested in comment in the file.

@github-actions
Copy link

github-actions bot commented Dec 1, 2021

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale Stale PR or inactive for long period of time. label Dec 1, 2021
Copy link
Contributor

@MaxwellDupre MaxwellDupre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed these are not new and should be added but could you add to test Lib/test/test_mimetypes.py please?

@kixorz
Copy link
Contributor Author

kixorz commented Feb 1, 2022

Thank you. I'll add the test.

@merwok
Copy link
Member

merwok commented Feb 1, 2022

On second thoughts, one ticket and one PR to add both types is fine.

@kixorz
Copy link
Contributor Author

kixorz commented Feb 1, 2022

I added the test.

@merwok
Copy link
Member

merwok commented Feb 2, 2022

@merwok merwok changed the title bpo-45639: Add modern image formats to mimetypes bpo-45639: Add webp and avif image formats to mimetypes Feb 2, 2022
@TeamSpen210
Copy link

If webp is nonstandard, shouldn't it go in the non-strict dict instead?

@kixorz
Copy link
Contributor Author

kixorz commented Feb 2, 2022

I think I added the news entry. Thanks for the guidance.

@ghost
Copy link

ghost commented May 3, 2022

All commit authors signed the Contributor License Agreement.
CLA signed

Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, and sorry this was left hanging for so long! We still have a few days until the 3.11 feature freeze, and I'd love to get this in.

A few small things:

  • You'll need to resign the CLA with our new process
  • As @TeamSpen210 says, webp is nonstandard and should be in the common_types list.
  • The docs build is failing because the blurb is missing a trailing newline.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@kixorz
Copy link
Contributor Author

kixorz commented May 3, 2022

I have made the requested changes; please review again.

@bedevere-bot
Copy link

Thanks for making the requested changes!

@JelleZijlstra: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from JelleZijlstra May 3, 2022 17:54
@JelleZijlstra
Copy link
Member

@kixorz thanks for the change! The tests are failing now, I think webp needs to move elsewhere in the test suite because it's not recognized by default.

@kixorz
Copy link
Contributor Author

kixorz commented May 3, 2022

I believe I fixed the tests.

@@ -95,13 +95,16 @@ def test_non_standard_types(self):
eq = self.assertEqual
# First try strict
eq(self.db.guess_type('foo.xul', strict=True), (None, None))
eq(self.db.guess_type('foo.xul', strict=True), (None, None))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is duplicated

Suggested change
eq(self.db.guess_type('foo.xul', strict=True), (None, None))

@kixorz
Copy link
Contributor Author

kixorz commented May 3, 2022

Thanks for checking it. Sorry, I missed it. Should be fine now.

@JelleZijlstra JelleZijlstra merged commit 6dee695 into python:main May 3, 2022
@kixorz kixorz deleted the modern_mime_imagetypes branch May 3, 2022 21:41
@kixorz
Copy link
Contributor Author

kixorz commented May 4, 2022

Thanks for merging my changes!

kidanger added a commit to ipol-journal/ipolDevel that referenced this pull request May 7, 2022
Changes were performed on integration and core.

Resources:
- https://www.iana.org/assignments/media-types/media-types.xhtml webp
  is still not listed there
- python/cpython#29259 was merged 3 days ago
MooseyAnon added a commit to MooseyAnon/corna that referenced this pull request Jun 19, 2024
It seems that support for detecting the mimetype of webp
file formats was added to pythons mimetype module in the
python3.11 release [1], as result we skip the webp media test.

The skipping is done via a conditional so will automatically run
once we update to python3.11 without having to change anything.

We also add a few extra comments around using a more robust file
type detection method (like libmagic), there are some links for
reading when the time comes to fix this, its not very pressing atm.

[1] python/cpython#29259 (review)
MooseyAnon added a commit to MooseyAnon/corna that referenced this pull request Jun 19, 2024
It seems that support for detecting the mimetype of webp file formats was
added to pythons mimetype module in the python3.11 release [1], as result we
skip the webp media test.

The skipping is done via a conditional so will automatically run once we update
to python3.11 without having to change anything.

We also separate gif tests from webp tests by adding a new test that uses a
a gif with a `.gif` extension instead of the previous `.webp` extension.

We also add a few extra comments around using a more robust file type detection
method (like libmagic), there are some links for reading when the time comes to
fix this, its not very pressing atm.

[1] python/cpython#29259 (review)
MooseyAnon added a commit to MooseyAnon/corna that referenced this pull request Jun 19, 2024
It seems that support for detecting the mimetype of webp file formats was
added to pythons mimetype module in the python3.11 release [1], as result we
skip the webp media test.

The skipping is done via a conditional so will automatically run once we update
to python3.11 without having to change anything.

We also separate gif tests from webp tests by adding a new test that uses a
a gif with a `.gif` extension instead of the previous `.webp` extension.

We also add a few extra comments around using a more robust file type detection
method (like libmagic), there are some links for reading when the time comes to
fix this, its not very pressing atm.

[1] python/cpython#29259 (review)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Stale PR or inactive for long period of time.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants