It is indeed the case that pixelated is mostly only useful if you're scaling by integer multiples.

So the question then is - are you doing something that really wants pixelated-style scaling, but centers on non-integer scale multiples? If so, can you provide more detail on this?

(There's a lot we could potentially do to make scaling different or better, but it needs to be driven by a demonstrated need that we have reason to believe is at least somewhat common, or uncommon but highly valuable for the small population that needs it and very painful to work around.)

So the question then is - are you doing something that really wants pixelated-style scaling, but centers on non-integer scale multiples? If so, can you provide more detail on this?

Sure. The problem is that in many cases, it is difficult to determine whether the scale factor is integer or non-integer at design time. The first problem is screen resolution: Websites are usually designed so that elements are the same real size on different screens, but that means what is 200% on a high-resolution screen may become 150% on a lower-resolution screen. Plus, there are some high-resolution screens with non-integer dppx (device pixels per CSS pixel), so even CSS pixels can be trusted.

Also, when I want to get a better look at an image (such as a retro game screenshot), I may manually zoom in on it in the browser. Therefore, I want even images that are normally displayed at 150% to have the image-rendering:pixelated property if they would look best pixelated at 400%.

So the question then is - are you doing something that really wants pixelated-style scaling, but centers on non-integer scale multiples? If so, can you provide more detail on this?

Oh, this is timely. I've just run into trouble with "pixelated" and a window.devicePixelRatio of 1.5 on my desktop pc.

(cf increpare/PuzzleScript#568 )

[If you have a desktop pc with non-integral devicePixelRatio you can repro it yourself here https://www.puzzlescript.net/play.html?p=fd4e3445b63068676f72 by resizing the window down]

You can see some horizontal lines like on the "S" of "simple block pushing game" are thick, while the "start game" lines are thin, even though both are one pixel high.

The behaviour I'd like to have available to me for the purpose of a game engine is for it to display the biggest possible image that will fit within the bounds of the canvas, centered inside it, where all upscaled pixels are the same size. (It's a pretty standard approach).

I've tried many different approaches to trying to get it to display "integer-multiple"-pixelated but they number of different display surfaces and zooming ratios that constitute the browser rendering stack make this feel like an impossible task.

The behaviour I'd like to have available to me for the purpose of a game engine is for it to display the biggest possible image that will fit within the bounds of the canvas, centered inside it, where all upscaled pixels are the same size. (It's a pretty standard approach).

That's different from what's being asked about here, and is probably a good fit for an object-fit value, probably as a modifier alongside cover/contain. Then you could apply object-position: center to position it as you described. Could you raise this as a separate issue?

Okay, so the proposed behavior is to scale up to the nearest integer multiple of the natural size that doesn't overshoot the actual scaling factor, using NN to maintain pixelation, then use auto scaling to finish reaching the desired scale factor. This sounds pretty reasonable, agenda+ to discuss.

Important point for discussion - would this be okay to just build in as the behavior of pixelated? Especially on half-scales like 1.5x or 2.5x the NN behavior already looks bad; would it be better to default these cases into being slightly smoothed? Or should we keep pixelated as strictly NN, and add a variant keyword for this behavior? I lean toward baking this into pixelated.

Could you raise this as a separate issue?

I'll give it a shot, thanks for the suggestion.

Or should we keep pixelated as strictly NN, and add a variant keyword for this behavior? I lean toward baking this into pixelated.

For what it's worth I'd prefer if pixelated meant pixelated, even if resulted in weird artefacts like in the first post. There are so many ways to smoothly interpolate pixels in css, but only one way to keep them 'crisp'.

Personally, I agree with baking this into pixelated. Yes, pixelated should mean pixelated, but I don't think nearest neighbor interpolation with 150% scaling ratio looks pixelated: Squares that vary in size from 1*1 to 2*2 do not look like pixels. I think it would be better to add a new keyword nearest-neighbor, which means nearest neighbor.

The CSS Working Group just discussed [css-images-3] image-rendering:pixelated should not force "nearest neighbor" (or similar) when the scale factor is far from an integer (e.g. 150%), and agreed to the following:

• RESOLVED: Bake the smoothing into non-int changes in current pixelated value. Add a new value for nearest neighbor jaggedness

Clarification: would downscaling always use linear scaling since the nearest integer is 1? I know nothing is "pixelated" in that case, but would still be nice to know

Yes, that's how I'm writing the spec; it makes sure that you find the nearest integer multiple greater than zero to NN it to, then you use smooth scaling to take it the rest of the way, so downscaling is always smooth-only.

(It doesn't have to bilinear scaled, you can use whatever algo you want to take it to the finish line.)

Some relevant history: we had a csswg resolution a while ago, back in https://lists.w3.org/Archives/Public/www-style/2014Sep/0384.html , to allow "pixelated" to use a better scaling algorithm specifically when downscaling (i.e. for scale factors < 1.0). Though this edit didn't make it into the spec (yet).

I think the feature-request here (and today's csswg resolution) is just an elegant generalization of that earlier resolution, to now also suggest (mandate?) a better scaling algorithm for fractional values that are larger than 1 as well -- not just fractional values that are less than 1. The previously-agreed-upon < 1.0 case is just a special case of the behavior that we resolved on today.

(It doesn't have to bilinear scaled, you can use whatever algo you want to take it to the finish line.)

Bicubic or Lanczos would be better than bilinear.

For posterity, and to express my "this will add cost" concern a bit better: I think a viable implementation here is a two step process. First, upscale using NN to nearest integer multiple. Then, use something like bilerp to take it all the way to the right scale. The problem (at least in Chromium) is that we would need to store the intermediate result in memory before doing bilerp sampling.

This means that if we have something like 1,000,001% scale of an image that is 150x150, previously we'd be able to sample directly out of the 150x150 image and obviously only sample where needed so that after all the clipping, we'd end up with the correct result. However, for this case the result should be a size 1500001.5 image (if my math is right), which means the new algorithm would say that we need to use NN up to 1500001x1500001 and then use bilerp. That's not something we can store, even temporarily :). I know that this is a contrived example, but I think there are less contrived examples that would suffer from similar problems.

I'm also hoping there's a way to just sample out of the original image (150x150) with some clever math that would effectively end up with this algorithm, but I haven't really thought about it at length yet.

Yes, NN can be trivially done with a single source pixel of margin around the window that you're scaling. If we can currently handle auto-scaling a 150x150 image to 150e6x150e6 (presumably by only scaling a window of the source), we can handle doing the same with an NN step first.

Here is an idea for doing all of this in one pass.

https://demos.skia.org/demo/up_scaling/

The top row shows the suggested one-pass technique: custom shader performing 4 samples per pixel
The bottom row uses two passes: upscale by int amount, and then bilerp the rest of the way

Some points by a long-term pixel-perfect integer-scaling enthusiast, a six-year owner of a 4K monitor used at 200%, and the author of the article about integer scaling and of the SmartUpscale browser extension (the latter tries to do exactly what this proposal is about; has ~800+ users on Firefox and 2000+ users on Chrome):

1. The ability to apply nonblurry scaling at non-integer scales should work for background images too. This would allow applying this approach to multiple background layers, which is currently almost impossible to achieve in a reasonable way.

2. There should be a way to choose a specific scaling algorithm at least for non-integer scales, but preferably for each of both cases. I planned to propose something like this for a while, and this is a possible syntax I thought of:

   image-rendering: integer(nearest) fractional(bilinear);

   So we would have a pair of value functions, each accepting a keyword corresponding to the specific desired algorithm for each of the two cases: integer scales and fractional scales.

3. This is probably obvious, but just in case: the feature should be tied to physical pixels. It should not matter how exactly the specific element is scaled on the page, including HTML attributes (width/height for IMG), CSS declarations like max-width: 100%, and CSS transforms (e.g. transform: scale(2)).

4. A considerable example of using the hybrid approach with applying nonblurry upscaling at integer scales and a blurry averaging one at fractional scales is DPI scaling (DPI virtualization) in Windows 10. It upscales user interface of applications that are either not declared as DPI-aware or forced by the user to be considered as non-DPI-aware.

5. (This part might be postponed to a next spec level/version.)
   It might be useful to have a way to limit the maximum scale of the element that nonblurry scaling is applied at. In case of non-pixel-art images, nonblurry upscaling makes main sense at scales equal to or lower than the pixel density that may be considered as a sort of threshold for single pixels to get noticeable as squares.

   For example, I use a 24-inch 4K monitor at 200% OS-level zoom, so my pixel density (devicePixelRatio in terms of web development) is 2.0, and 2×2 solid-color square pixels are almost indistinguishable as squares when using integer scaling with 3D content, photos or videos (e.g. when displaying 1920×1080 aka Full HD in full screen). But at 300% (3.0) scale (e.g. when scaling 720p→4K), 3×3 pixels get quite noticeable as squares, so a way to trigger blurry scaling would be useful to prevent visible pixelation at too high scale levels while using integer scaling solely for the purpose of preventing unreasonable blur when logical pixels-squares are almost indistinguishable.

   Using the possible syntax I provided above, this could be expressed as an optional function inside the integer() function:

   image-rendering: integer(nearest max-pixel-density(system)) fractional(bilinear);

   max-pixel-density(system) would mean that image-like elements (images, background images, videos, canvas elements) should be scaled with no blur at integer scales up to the OS-level zoom inclusive and scaled with blur otherwise. For example:

   • In case of OS-level zoom of 200%, integer scaling would only be applied at 200%.
   • In case of OS-level zoom of 400% (e.g. with 24-27-inch 8K monitors possible in the foreseeable future), integer scaling would be applied at 200%, 300%, and 400%.

   For what it’s worth, such a limit can be specified in the SmartUpscale extension mentioned above, via the “Maximum image zoom that blur should be prevented at” setting which accepts integer numbers.

   We could probably imagine an extended syntax like max-pixel-density(system+1) for more flexibility, but it’s probably enough for now.

Thanks.

mikerreed's sharpened bilerp is pretty cool. It's kinda similar to the other nearest-neighbor-but-also-bilinear shader -- a somewhat common requirement, it seems.