Is WebP really better than JPEG?

If you have used tools like Google’s PageSpeed Insights, you probably have run into a suggestion to use “next-gen image formats”, namely Google’s WebP image format. Google claims that their WebP format is 25 – 34% smaller than JPEG at equivalent quality.

When testing out WebP using a perceptual image optimizer, I ran into a peculiar issue: the WebP files were of very similar size compared to compressed JPEGs, in many cases larger. I’m not only one who noticed this, but Mozilla also noted in their 2013 study that WebP doesn’t generally have much better compression efficiency when compared to JPEG. (Note that Mozilla somewhat walked back from this and implemented WebP support for Firefox in 2019)

I think Google’s result of 25-34% smaller files is mostly caused by the fact that they compared their WebP encoder to the JPEG reference implementation, Independent JPEG Group’s cjpeg, not Mozilla’s improved MozJPEG encoder. I decided to run some tests to see how cjpeg, MozJPEG and WebP compare. I also tested the new AVIF format, based on the open AV1 video codec. AVIF support is already in Firefox behind a flag and should be coming soon to Chrome if this ticket is to be believed.

Images and Tools

For the testing I used the Kodak image dataset in 3 different sizes: 500 px, 1000px and 1500px.

• For JPEG conversion I used cjpeg with --optimize flag, --progressive flag and 4:2:0 chroma subsampling.
• For MozJPEG conversion I used MozJPEG with --optimize flag, --progressive flag and 4:2:0 chroma subsampling
• For WebP I used cwebp with -m 6 flag for maximum compression and -af for auto filter which presumably trades encoding time for increased quality. WebP only supports 4:2:0 subsampling so this doesn’t need to be specified separately.
• For AVIF I used colorist with flags --tonemap off, --yuv 420 and --speed 0 which is the slowest but highest quality encoding

In addition to these, ImageMagick was used to scale down the images from the originals and converting between PNG, WebP and TGA (cwebp only supports TGA input). All conversions were done in sRGB color space.

For comparing the quality I used kornelski’s dssim utility which calculates structural similarity index between images. My target SSIM is 0.0044 which roughly corresponds to JPEG quality of 85.

Results for 500px images

Open comparison in a new window

Here are the results on a graph:

If we look at the median file sizes, we can see that compared to cjpeg, MozJPEG is roughly 11% smaller, WebP is 18% smaller compared and AVIF is 31% smaller at the equivalent SSIM index.

Results for 1000px images

Open comparison in a new window

Here are the results on a graph:

If we look at the median file sizes, we can see that compared to cjpeg, MozJPEG is roughly 11% smaller, WebP is also 11% smaller compared and avif is 28% smaller at the equivalent SSIM index.

Results for 1500px images

Open comparison in a new window

Here are the results on a graph:

If we look at the median file sizes, we can see that compared to cjpeg, MozJPEG is roughly 9% smaller, WebP is the same size as cjpeg and AVIF is 28% smaller at the equivalent SSIM index.

Average for all image sizes

Just for fun, I graphed the averages of all image sizes. I know this might not be a fair comparison but still, here you go:

As you can see, when compared to cjpeg, MozJPEG is about 9% smaller, WebP is 6% smaller and AVIF is 30% smaller.

If you are interested, you can view the raw data for all the images as a spreadsheet here. Source code for the comparison app and raw images are available on GitHub: 500px, 1000px and 1500px. Check the originals directory for the raw images.

Conclusions

Is WebP better than JPEG?

So, is WebP better than JPEG? It depends if you are using the reference libjpeg library or the improved MozJPEG encoder.

WebP seems to have about 10% better compression compared to libjpeg in most cases, except with 1500px images where the compression is about equal.

However, when compared to MozJPEG, WebP only performs better with small 500px images. With other image sizes the compression is equal or worse.

I think MozJPEG is the clear winner here with consistently about 10% better compression than libjpeg.

Since most of the time WebP is used alongside JPEG fallback, by using WebP you will essentially double your storage costs with little benefit. So, in the end, I would recommend using WebP in only the following cases:

• You have a lot of small images in the 500 px range.
• You can’t use MozJPEG.
• You pick an arbitrary fixed quality instead of using a metric like SSIM.

In any case, when converting images to WebP, check that they are actually smaller than the JPEG equivalent. There’s no need to serve larger images to your users than needed.

How do image formats derived from video codecs differ from JPEG?

One notable difference between JPEG encoders compared to WebP (based on VP8) and AVIF (based on AV1) is that it’s pretty easy to see how the latter were derived from video codecs. JPEG compression uses the same quantization factor for each 16×16 “macroblock” so the compression is consistent throughout the image.

WebP and AVIF on the other hand use different compression factors for different parts of the image so while the detailed parts of the image retain their quality, surfaces like skin or the sky which have low detail are “smoothed out”. This is especially noticeable with the red window shutters in this image.

While the bricks in the image look sharp, the doors look almost like they have a “Smart blur” Photoshop filter applied to them.

I think this kind of adaptive compression is a valuable thing to have. Think about a photo with a forest and the sky. A traditional image encoder would have to decide a single compression ratio for the whole image. While it’s good to use a lot of bits for the forest trees with high-frequency detail, they are wasted for the sky with low-frequency detail.

A smarter encoder like WebP or AVIF will be able to process these areas separately to use the available bits efficiently.

Is AVIF the future of image formats?

I think AVIF is a really exciting development and compared to WebP it seems like a true next-generation codec with about 30% better compression ratio compared to libjpeg. Only concern I have is the excessive blurring of low detail areas. It remains to be seen if this can be improved when more advanced tooling becomes available.

Right now the tooling is a bit spotty. Colorist was the only program I found which can reliably encode AVIF files. Encoding AVIF files is also really slow! A big image can take several minutes to encode. I’m using the AOM encoder but rav1e might be faster. Browser support also still in progress. Firefox has AVIF support but it’s behind a flag and it doesn’t seem to read ICC profiles correctly. Still, it’s more browser support than Apple’s “next-gen” HEIF which isn’t even supported in Safari.

I think in the next year or so we might see a radically different landscape. With Chrome on board, we could see supported browsers jump to something like 70% of all browsers which means AVIF would be a pragmatic thing to support in web projects.

Caveats

In this test, I only used photographic images. WebP may be better when compressing graphics, for example, since it supports lossy compression for the alpha channel which PNG and JPEG do not.

I also tested the images in “Web quality” target of 85 so WebP may perform differently in very high or very low-quality settings.

Also, Google’s study used a different program to compute the SSIM values. In my tests, I used the dssim utility which computes multi-scale SSIM in LAB color space while the former seems to use simple SSIM in RGB color space. This means they likely give different results but it’s hard to say which is more accurate.

Edit 21.6.2020

By popular demand, added error bars to the graphs. These represent 25th and 75th percentiles. I also added GitHub links for the comparison app and raw images.