Compression art: visualizing the obscured

Demolish the eerie ▼oid. Music composed by Extraboy.
Extraboy used a rain vst, PIPPO vst and C64 noise to compose a complimentary soundscape.

████ Infinitely pliable, yet holding shape █
Especially over the last century communication has been striving to become increasingly fast and transparent. As a result the concept of immediacy has become one of the key words of this period (at least for media theoreticians).
To answer this wish for acceleration the transfer of information and information objects themselves have been forced to change. Digital compression technologies are used to encode files into fewer bits (or other information-bearing units), to travel faster and further. These compressions reorganize the relations inside and between the data of for instance sounds and images by means of scaling, reordering, and decomposing. Compressions also reorganize the context, time and space of information; basic values on which communication used to depend.

█ Compression █
The language or taxonomy of digital compressions is horribly complex. Looking at a file, first of all there is the file format, which is a ‘container’ of for instance sound and image (think about mpeg, avi or mov). This container possesses the meta-information about what type(s) of compression-decompression protocol(s) - the codecs - are needed to store and transfer the information or to view the data object. This container however does not posses the compression algorithm itself (.avi is thus not a compression but only a container!).
In the second place, a file depends on the specific compression-decompression protocols (codecs). These protocols can in their turn be divided into two groups: lossy and lossless. Lossless compressed files can be rebuilt exactly the way they were before they were compressed; they have retained all information while being compressed. But while in the beginning the development of new audio and video technologies were focusing on the improvement of sonic quality, by for instance the reduction of noise, recent developments seem to lead to a reversal of this trend. Lossy compression has become almost ubiquitous, whereas original (RAW and lossless compressed) information is mostly absent and relatively obese, especially in the realms of digital music, photography and cinema.
New lossy data compression technologies, such as the mp3 data format have made it possible to distribute music easily, but in lower quality than the CD. Lossy data compression takes a distance from the original file. This compression focuses only on the data that is important for the eye and/or the ear to perceive, and discards the information that is believed to be of lesser importance. For video images for instance, perception depends on the thresholds of luminance (brightness) and chrominance (coloring) in space and time. Therefore, video codecs are based on and centered around the transmission of these two values as efficiently as possible.
Most compressions and codecs are relatively concealed; they rarely come to the surface to reveal their true nature - their system of rules. They have been build and rebuild a thousand or more times to be as transparent and invisible as possible. However, as I also wrote in the Glitch Studies Manifesto, the dominant, continuing search for a noiseless channel has been, and will always be no more than a regrettable, ill-fated dogma.
Even though the constant search for complete transparency brings newer, ‘better’ media, every one of these new and improved techniques will always have their own fingerprints of imperfection. Thus, every compression technology will have its visible, or less visible artifacts, that sometimes come to the surface.

█ Compression art █
While some artists believe that lossy compressions harm their work, since these compressions essentially chop away pieces of information, other artists feel that compression is a necessary part of the character of their canvas. They believe that these rigorous forms of compression add certain artifacts that materialize the inherent character of the digital canvas; they (re-)visualize the obscured. The music label 20kbps for instance, is dedicated to low compression music releases; they advertise the extreme low of 20kbps instead of the already low-end musical standard of 128 kbps.
Johan Larsby also investigated the question of how to apply compression effects to audio. He came up with these results (1 and 2). I especially like the fast degradation by feeding back the compression into itself - reminds me of Seb's I am sitting in a room and Universes' digital decay experiments.
In 2002 Ana Kronschnabl released George The Mewvie (2002), which features her cat in extremely poor compression under the header “new films for new machines”. This film gives an impression of what the final film would look like, once it had been encoded and compressed for streaming through a 56K modem, which was the standard at that time. The three-minute long video shows a brown, blocked away moving and purring cat.
The RyBN audiovisual-art collective from France also makes use of lossy artifacts. They describe themselves as specialized in the “realization of installations, performances and interfaces by referring [...] to the codified systems of the artistic representation (painting, architecture, counter-cultures).” RyBNs Monochrome performance for instance, takes place in total darkness and consists of only black sequences extracted from various DIVX movies. The saturation parameter (color intensity) of the black video channel is linked to the sounds master volume. During the performance, the saturation becomes higher when the volume is turned up. This makes several blocks emerge from the previously black screen that obtain their form and dirty yellow greenish color from the DIVX compression codec.
Works like George The Mewvie and the Monochrome performance are not always aesthetically engaging; at times they only show very slow moving blobs. But this (lack of) visual pleasure fits well with the critique they voice towards the progression of digital technologies and the common opinion that digital communication is supposed to be fast, perfect and clean: the dogma of instant pleasure. In the Plugin Manifesto Kronschnabl and Tomas Rawlings instruct the reader to:

Use codecs and compression creatively: There are limits to film online. Size should be limited and there is always the ‘problem’ of compression. But the trick is to use this problem in a creative way and play with it.

Kronschnabel and Rawlings continue their argument by stating that the user should investigate the tools created for the internet and explore what he can do with these technologies in creative terms. Because, as the Camera and the celluloid defined the film in cinema, technologies like codecs and compression artifacts will define the material of the digital (internet) film.

█ Macroblock studies: research in the digital material of film. █
The lossy compressed video image is framed fundamentally different from analog or RAW video footage. First of all, the frames no longer rely upon raw pixels. Instead, macroblocks have become one of the elementary components of the lossy compressed moving image (at least under current standard codecs of, amongst others, the Moving Pictures Experts Group ‘MPEG’). Lossy compressed video often depends on luminance (brightness) and chrominance (coloring) thresholds arranged within 16x16 pixel (more or less) macroblocks within the keyframes (the I-frames) of an image sequence. The thresholds (or frequencies) of chrominance and luminance depend on an oscillating cosine function (following Fourier Transform).
Moreover, the material of the digital film is no longer based on a linear series of discrete images (a sequence); instead the video consists of different kinds of frames (I-frames or reference/key frames, P-frames or forward-predicted frames and B-frames or bi-directional frames), of which only the keyframe possesses a complete matrix of macroblocks. The frames between the keyframes (the P- and B-frames) consist of motion vectors that index only the difference in position (the offset) of the macroblocks between the original and the next frame. The handling of space and time within the video technologies is thus significantly different between the linear analog or RAW footage and lossy compressed footage.
Just like any other technology does, these new digital technologies posses their own specific artifacts. A couple of these artifacts are for instance “macroblocking”, “mosaicking”, “pixelating”, “quilting” or “checkerboarding”. And as is normal with any new technology, there is a growing group of (video) artist exploring these different artifacts of this new and constantly changing digital canvas.
A recently popularized wave of video artworks was based on the deletion of keyframes and the exploitation of the vector motion of P-frames. I described this current, which is dubbed datamoshing, pixel bleeding or just compression art, in some earlier posts.
A bit more obscure (partially because maybe the results are a bit less magical) is a related strategy that doesn’t rely on the deletion of keyframes and the exploitation of motion vectors, but on the exploitation/bending of chrominance and luminance algorithms of the Fourier Transform thresholds of the macroblocks. This method visualizes the otherwise mostly obscured macroblock structures of digital video.
The non-technical term macroblocking thus refers to compression/noise artifacts in lossy compressions, that shows up when for instance some data of chrominance/luminance is broken, the FTT cosine function itself is bended or some video image data is for instance edited in a hexeditor.

███ Demolish the eerie ▼oid █
In "Demolish the eerie ▼oid" I exploit macroblocking, by opening a DV file in texteditor and playing around with the code.
What became clear to me is that multiple things can happen – and I am not sure (yet) what pieces of code trigger what results. In one of the first tries I encountered some quite mystifying, but nice new organizations of color (chrominance) structures. The matrix of macroblocks is still completely intact but the chrominance values are ... off.

The structure of the video is still intact - you can still recognize me when you look very close/pay attention. The extreme color shift make it hard though.

The matrix of macroblocks holds some puzzling but pleasantly surprising opportunities. It can also be rearranged.
But most of the bends have the more usual outcome: a carnival of macroblocks and an ambient landscape of noise, flickering back and forth, confused and agitated about its proportions.And to finish a dessert of mess: when I open the file in VLC player, I got some nice RAM glitches when change the size of the videoscreen.
The data of the file is so high that the player can only show the image with parts of what is still in the buffer.

██████
Kronschnabl, Ana, and Thomas Rawlings. Plug in Turn on: A Guide to Internet Filmmaking. London: Marion Boyars Publishers Ltd., 2004.
Mackenzie, Adrian. “Codecs.'' In: Matthew Fuller, ed. Software Studies. Massachusetts: MIT Press, 2008. p. 48-55. p. 36
Mackenzie, Adrian. "Every thing thinks: sub-representative differences in digital video codecs". Institute for Cultural Research
Mackenzie, Adrian. "Codecs: Encoding/decoding images and sounds"

The music label 20kbps can be found here
“ARTKILLART / CIMATICS.BE.” AKA/CIMATICS.
monochromatiq. “MONOCHROME.” INCIDENT.NET | MONOCHROME |
TUBE:MONOCHROME | RYBN.ORG.

Special thanks to Emilio Pozzolini, euphotic/Al Astakhov and Extraboy/Anders Carlsson.