• mina86.com

  • Categories
  • Code
  • Contact
  • Save 22% of your storage with this one easy trick

    If you’re a digital archivist, photography enthusiast or data hoarder, you’ve likely faced the ‘No space left on device’ error. Before you start deleting your files, there’s a way to reclaim space without losing a single pixel: JPEG XL. Unlike typical lossy transcoding which degrades quality, JPEG XL allows for bit-for-bit reconstruction of the original files.

    JPEG XL is the newest image file format developed by the Joint Photographic Experts Group with, among other features, better compression. Normally converting between formats with lossy compression is inadvisable as it exacerbates quality loss; however, JPEG XL offers lossless JPEG transcoding.

    Unlike normal conversion (e.g. from JPEG to AVIF), JPEG transcoding doesn't re-encode the image data. It merely repackages the existing information so that the original JPEG file can be recreated bit-for-bit. And all that with 22% better compression.

    Migration

    The conversion is done with the cjxl tool distributed alongside the reference JPEG XL implementation. (On Debian it’s included in libjxl-tools package). It’s invoked as follows:

    cjxl -j 1 -e 10 -d 0 image.jpg image.jxl
    • The -j 1 flag is the ‘magic’ that enables the lossless transcoding.
    • The -e 10 specifies the highest compression effort. This does not affect the quality of the image; instead, it trades encoding time for better compression ratio.
    • The -d 0 enables lossless compression (see below) if the source file is not JPEG.

    To verify the perfect reconstruction the djxl tool together with cmp program can be used:

    djxl image.jxl decoded-image.jpg
    cmp image.jpg decoded-image.jpg ||
        echo Reconstruction produced different JPEG file.

    To migrate several files the jpg-to-jxl.py script can be used. It takes files to migrate as arguments and, upon successfully conversion, deletes the source. It can be paired with GNU Parallel to utilise all available CPU cores. For example, the following converts all files in /path/to/gallery directory:

    find /path/to/gallery \
         \( -name \*.jpg -o -name \*.jpeg \) -print0 |
        parallel --null nice python3 /path/to/jpeg-to-jxl.py

    Kliuchnikov et al claim 22% file size reduction which is in line with my own test on over 400 thousand images.

    Notable limitations

    It’s not all sunshine and rainbows. JPEG XL is new and some tools still do not recognise it. Even if the format is decoded, some of its features may be mishandled.1 Sometimes this can be resolved by installing additional tools. For example, an official JPEG XL plugin is available for Windows 11 and a third-party JXLook Quick Look plugin for macOS.

    Support is practically non-existent on the web. Only Safari handles basic elements of the format and infamously Chrome has removed its experimental support. The best one can do on the web is to use JPEG XL for storage but reconstruct JPEG files on the fly when a user fetches the image.

    It’s also worth pointing out that the encoder may fail to handle some malformed JPEG files. cjxl can be run with additional --allow_jpeg_reconstruction 0 flag to force the conversion, although byte-for-byte reconstruction will no longer be possible in that case.

    JPEG is lossless now‽

    But there’s more. JPEG XL also comes with lossless image compression. That is, it offers: i) lossy image compression (like JPEG), ii) lossless image compression (like PNG) and iii) lossless JPEG transcoding (described above). To use the lossless image compression, -d 0 argument is needed.

    However, JPEG XL does not provide any indication or a flag whether a file used lossy or lossless compression and authors of the format are hostile towards the idea. For me it’s important to know whether the image comes from a lossy or lossless source. Because of that, for lossless compression I’ve stuck to the WebP file format.2

    My current practice and recommendation is to transcode all JPEG files to JPEG XL and convert all lossless images to WebP.

    FormatTotal sizeSaving vs. PNG
    PNG (zopflipng -m)9.60 GB
    WebP7.63 GB~20%
    JPEG XL6.39 GB~33%
    Lossless compression file size comparison tested on 1346 images.

    1 For example, while Geeqie decodes JPEG XL files just fine, it does not apply the embedded colour profile. To help with testing, the jxl-icc-test.tar.gz archive contains example JPEG XL file with a expected good and bad results. 

    2 WebP supports lossy and lossless compression just like JPEG XL, however the difference is that it’s possible to determine the format by reading the file.