Stylistic Image Compression

Having browsed the web, you may have noticed that there are a lot of images. As with all things online, images come with a cost – in terms of energy, money and time – from the storage and servers needed, to transmission, and the power required to download and display content on our devices. As we go about minimizing the footprint of our websites, addressing the impact of images is one of the things that need to be considered.

The annual report from HTTP Archives sheds light on the topic¹. The report, which is based on a sampling of millions of websites, concludes that the average weight of a website largely depends on image use. As Tom Greenwood puts it in his book Sustainable Web Design, “Roughly half of this data transfer is image files, making images the single biggest source of carbon emissions on the average website”². While many other elements on websites can be streamlined, compressed and optimized, images are far larger and bulkier, contributing to a large weight on any website.

The photos we store on our phones and computers are several megabytes in size, but online images have to be much smaller in order to be stored and transmitted efficiently and fast. It is here image compression becomes essential. Most platforms and tools we use to build websites today compress images automatically as they get uploaded. The responsibility is normally in the hands of the developers. Therefore there is no incentive for designers to invest time or effort in considering how image size, formatting and compression impacts the weight of a website. Although developers continue to find new technical solutions and image compression becomes increasingly efficient, the level to which we can compress our images is ultimately dependent on what our eyeballs deem acceptable. How far we can compress our images is therefore not limited by technology itself, but ultimately a question of aesthetics.

By recognizing aesthetics as a limiting factor, the problem can be turned on its head. For designers, compression as a constraint could be used to achieve specific visual qualities, while also reducing the weight of websites. If we decide to see compression as a visual design tool, something more than just a technical means to an end, we could hit two birds with one stone.

Seeing this potential was the outset of my exploration into the world of image compression. From experimenting with a range of different formats, expressions and styles, I have identified some of the possibilities and challenges such an approach would entail, and reflected on how they could fit into our practice as designers.

Dithering

Dithering originates from to the early days of computing, when devices were more limited in what they could store and display. Dither was used as a technique in order to create the illusion of a larger range of color, while preserving smaller file sizes. Dithering is in many ways a continuation of a long tradition of techniques seen in print, graphics and art - a digital successor to methods such as halftone, stippling and crosshatching.

Dithering as a method of reducing file size is still essential in a wide range of applications today, ranging from signal processing to sound and video, but is rarer in images, where displays and hard drives have gotten better and larger instead. Submerged from our collective perception, dithered images occasionally surface to our attention, typically in the form of GIFs and memes.

The look of dithered images usually conjure a sense of nostalgia back to the earlier days of computing. For designers seeking to create a retro look, this can be used with great effect. But the historical background also makes it challenging for designers to utilize it for more contemporary contexts. I was curious to explore ways of use dithering in creating a contemporary styles, instead of as an allusion to the days of arcade games and 90s clipart (which has become somewhat trendy today).

Part of the motivation came from my observation that dither was not consistent across monitors and had a profoundly different look on higher resolution displays. As screen resolutions have increased over the years, the blocky, pixely look that is commonly associated with anything retro is increasingly rare, as pixel sizes have become so small that it’s square nature is no longer perceivable. What used to be a rigid grid of squares is changing character, looking more akin to fine dust or sand. This fundamentally changes the visual look and feel of dithers, with an emphasis on its textural qualities. This alone should justify a reevaluation of its visual and aesthetic qualities. As with any medium or technique, from the grain of analog film to the earthy pigments of oil paints, dithers have their own unique visual qualities, characteristics and quirks. What sets it apart is its almost tactile, could I say sensual, textural quality it brings to images.

There are many common and established variations for producing different styles of dither. It should be noted that there is no wrong way. Methods of dither is an aesthetic choice, open for different technical uses, styles and expressions. In my experiments I largely based the work on established methods, exploring new ways to use and nudge them to achieve different styles.

To my dismay, it was more challenging as expected to overcome the retro looks. In many instances I ended up with something even more retro than what I started with! I choose to blame my own failings rather than some inherent quality to dither in itself. Although my experiments failed to achieve my initial desired goals and aspiration, I strongly hold on to my conviction that dithering could be used to great effect in the future.

Working with these experiments also made me familiar with some of the drawbacks and challenges of dithered images. Arguably, one of the biggest challenges is that dithered images produce moiré patterns when image pixels don't align with screen pixels. This is rarely a wanted effect (even if it produces some interesting effects). Scaling up images on the contrary, reveals the blocky, pixely nature of these images. The interpolation between scaled up pixels creates mushy gradients between pixels that are usually less noticeable in other image formats. As images can’t be scaled arbitrarily to fit any screen resolution or size, designing with dithered images pose some challenges for how images should respond to dynamic layouts, varying screen sizes and resolutions. This adds some extra challenges when working with dithering that are not an issue with displaying images conventionally.

Lastly, it should be noted that although other formats such as JPEG and WebP outperform formats that employ dither in terms of quality and compression, the dither still stands out as a tool with aesthetic potential in its ambiguity. Rather than dismissing dithering for its challenges, I would argue that there is uncovered ground for future explorations.

SVG Filters

SVG filters were originally a means to apply effects to SVG vector graphics, but can be used to extend the functionality provided by CSS to all web page elements. SVG filters consist of a set of 16 so-called filter primitives. These are common graphical operations, such as applying blur, distorting the image, and ways of manipulating colors. Although the set is fairly limited, SVG filters can be combined, leading to a surprising amount of interesting effects and visual outcomes. Although nowhere as powerful as more advanced tech (such as WebGL), SVG filters have their advantages.

Firstly, SVG filters are quite simple and easy to work with. They are light, as they require no boilerplate. They have broad support across browsers and work on most devices, including older ones. Additionally, they support simple animations, and are easy to make interactive when combined with Javascript. Although nowhere as powerful as shaders, they are in some ways the poor man's alternative.

I was interested to see how these effects could be used in combination with image compression. By replacing some of the large data sizes required for images with smaller pieces of code, I wanted to explore ways to achieve interesting styles that go well with reduced image sizes. Assuming images were compressed further, SVG filters could provide ways of “decompressing” images on the client’s end, creating light, ‘arty’ low-res images that could communicate and be as visually pleasing as heavy, sharp hi-res images.

When I tested the SVG filter experiments on different browsers they ran noticeably slow, excruciatingly so at times. The larger the area the effects covered, the slower it ran, which suggests these effects are probably not optimized or handled by the processing wonders of the GPU. From my limited understanding on these things, this is largely a result of how major browsers have implemented them. Sadly as a result, this makes them unviable for most serious uses, especially for animation and interactivity. As browsers are an old school piece of technology, it is questionable whether browsers will be improved or changed to better incorporate SVG filters. This in turn, will depend on whether browser developers see a demand or interest in their use, which, given the unpredictability of web trends, is impossible to say.

JPEG

JPEGs are one of the world's most popular and recognized image formats. While there are newer, more optimized image formats for web use, JPEGs still constitute the most popular format online. In the following experiments, I was curious to see if I could use the characteristics of JPEG compression, as a starting point for a stylized look.

Reflections

Working with this topic has been challenging. Compression and image formats are big topics, requiring understanding from across many disciplines and domains, from computational and algorithmic thinking, to math and color theory. Much of this lies beyond the comfort zone of a typical designer, myself included. These technical aspects stand as a large barrier. Even on a purely conceptual level, it is difficult bridging the gap between aesthetics and compression. While I see a lot of potential in such an approach, it is clear this is a challenging, and to some extent, extreme, approach to a problem.

But designers should not be afraid to approach this topic. Having a basic understanding of image formats and compressions can go a long way in informing and inspiring clever solutions and approaches to some of the problems we may face as we attempt to design more sustainably. Working with this has made me aware there are many possible avenues which are less dependent on technical mastery, and more about artistic vision through the creative use of images and small bits of CSS.

A stylistic approach to image compression would, by definition, not be generic to all contexts. So who would this be for? Could this be valuable for brands, businesses, publications, magazines, blogs and so on? How could it be used as part of their visual language and brand to create a unique visual style that stands out? Perhaps a website focused on long form essays on the movements of the skies could offer a nonintrusive, calm, almost ephemeral images to accompany its text, while a edgy, blog for canine street fashion could purposefully distort and manipulate their images for a raw, in–your–face effect. Those questions have been largely outside the scope of this project. As this project started out on the premise that a stylistic use of compression had not been explored as a design method, there was no telling what I would find, if anything at all. The next natural step would be to explore that further, through involving other parties, through concept development, evidencing and so forth.

Secondly, in choosing a stylistic approach, one would have to consider what the environmental footprint of such an approach would entail. Could the cost of designing and developing the methods outweigh the cost of use? How expensive is the computation involved in creating and displaying these images, compared to conventional methods? And how should we go about measuring the costs and benefits involved? Through my research and experiments, I have the understanding to carry this out at small scales, but without infrastructure and established parameters for feasibility assessments, we might find that production, development and maintenance could actually outweigh a traditional approach.