How to create compression artifacts [ Datamoshing ]

by mr datamosher


Vectorpunk [is also over according to mr O'Reilly]

[rest of this post is written with info I got from George Gardiner]
Although I just learned that Datamoshing is over, I want to quickly describe how you can make it happen, to late. So here we go,

When a video is encoded, each frame is stored as an I frame or a P/B frame. An I frame [Sven Konig calls them ∆-frames] is like a JPEG image, it holds the still image (that frame) in its entirety. P and B frames are the smart frames that allow videos to be compressed. They store only the differences between the current frame and the last frame. The effect you see in the datamosh videos is what happens when you store only the differences between frames, ie. when there are no I-Frame references.

Here is how you can do it the hardcore way:
1. Get your video in MPEG4 format, XViD or DIVX is fine.
2. Save it as an AVI file
3. Download a HEX editor and open the AVI file with it. (http://www.chmaas.handshake.de/delphi/freeware/xvi32/xvi32.htm is good, and free) You'll see a lot of numbers and symbols. This is the raw video data.
Each video frame in the AVI file will start with the code 00dc, you can search for these in the file (there will be a lot!). About 1 in 25 of these will be an I-Frame (though this depends on things like what quality you saved it at, etc). An I-Frame can be identified by the HEX string 00 01 B0 01 that will appear about 5 bytes after the 00dc frame start marker.
4. These are the frames you need to nuke. I think you can just hold delete until all the data is gone, and the next 00dc, or 00wb moves into its place.

Some notes:
1. NOT ALL VIDEO PLAYERS WILL LIKE YOU DOING THIS! It is a hack afterall, what you're really doing is breaking the video. After removing the I-Frames with this technique, the first thing you should do is use something like VLC to convert it back into an un-broken format. When you do this the visual effect you created by removing the I-Frames will be preserved, but VLC will insert new I-Frames so that all video players understand how to play it back (some throw a fit when they do not find the data they expect, others like VLC will carry on regardless). This is very important if you want to then upload to YouTube as it will probably not accept a broken video.
2. Should go without saying, but keep a backup of your original video before you start.
3. As you are removing video frames, you'll also loose audio sync over time. Best thing to do is probably put the audio on after you've finished.
4. You may get some interesting effects by keeping some I-Frames, eg. syncing the frames you keep to a beat or something?

[[edit"" this following method does no longer seem to work, probably because of a new version -> the youtube way does still work so try this instead!]]
Here is how you can do it the less hardcore way:
Use the program called VirtualDub. You need to use MPEG4, (DivX or XViD).
VirtualDub allows frame-by-frame direct editing, which allows you to delete the I-Frames (note: VirtualDub calls them Key Frames). As long as keyframe reconstruction is turned off you will produce the same effect with no nasty hex editing required. [[edit"" this method does no longer seem to work, probably because of a new version -> the youtube way does still work so try this instead!]]

There should be other ways that are even more easy (or hard), but these are the cool ways! Oh no, its over. We need something NEW !

From compression artifact to filter >> commodified effect!

An archeology of a compression filter. (or from noise artifact to filter)
A couple of months ago I was approached by a company that asked me for my knowledge on glitch and compression artifacts. They were especially interested in recreating bleeding pixels. I recognized their ideas in the work of Sven Konig and send them to his website (on which in 2005 already, Sven published a script that can be used to create such artifacts). Later the company send me an email that said they unfortunately cancelled their project.
I made my own video (and later another) though, and send it to Goto80, but in the end we also decided it seemed boring to just reuse a technique developed by somebody else - because it feels no longer as an experimental glitch, but more like an commodefied effect.
I kept track of the bleeding pixel effect and noticed its growing popularity, especially over the last few weeks. This lead me to write a small archeology of this particular compression artifact.

The bleeding pixel effect (or datamoshing) is located in a realm where compression artifacts and glitch artifacts and digital effects intertwine. The artifacts caused by compression are stable if triggered on purpose, but for the human eye quite unpredictable. Therefore, technically, they are not a glitch because the method is explainable and reproducible. They are however often perceived as a glitch (an error)- also because of their parallel existence within the realm of broken television signals and by artists used metaphorically as glitch art. The bleeding effect relies on deleting the information within one frame (the keyframe) which can be found once per 25 frames (more or less, depending on the frame rate) in a divx, xvid (or WMV) video.
I think the most interesting thing about using this effect is that it shows the materiality of digital film - it metaphorically translates the grain of the celloid to the digital pixel. Besides this, the what now has become an effect or commodity (most clearly because it has become an MTV aesthetic introduced by Kanye West), clearly lays bare the ever existing tension between what is a glitch and what is a reproducable glitch effect or commodity?
Here is a timeline. I know there are more works out there, but these works seem (for different reasons) more important or referenced too.


Sven Konig.
aPpRoPiRaTe!, 2005
Online distributed software script made to appropriate complete video files found in file sharing networks with minimal effort.

In 2007 Sven and Bitnik collaborative project Download Finished saw the light of day. Download Finished is a website that "transforms and re-publishes films from p2p networks and online archives. in Downloaf Finished, found footage becomes the rough material for the transformation machine, which translates the underlying data structure of the films onto the surface of the screen. the original images dissolve into pixels, thus making the hidden data structure visible. through Download Finished, file sharers become authors by re-interpreting their most beloved films."


Takeshi Murata
Monster Movie, 2005
Single channel digital video on DVD 4 minutes;
sound by Plate Tectonics. lo-fi version on Youtube


Paul B. Davis
(collaboration with Jacob Ciocci/PAPER RAD)
“Compression Study #2”, 2007
DVD projection (edition of 7), also on Youtube


David OReilly
Compression Reel, 2008
Vimeo Video


Data Mosher
Chairlift - "Evident Utensil" Music Video, 2009
Vimeo Video


Nabil Elderkin
KANYE WEST "Welcome To Heartbreak", 2009
Vimeo Video

Since the release of the last three videos (by O'reilly, Chairlift and Kanye), there has been a lot of back-and-forth comment-talk on who was the first to use this effect. The Kanye video will not even be released because "there is another video out there using the same effect". Even so, most of the video makes seem to be quite nice to me and didn't get me tired of the effect yet.


Rosa Menkman
Abandoned compression video project, 2009
Made with the help of Download Finished

Another couple of works that I really like but didnt index (due to lack of info) are by Antoine Catala and Cosmosabravo

I think it is only a matter of time before we will see that this artifact is developped into an browser script, like we have seen with much praised image-glitch plugins and browsers like glitchbrowser (2005/2008) developed by dimitre, ant scott, iman moradi.
Then this enchanting artifact will be changed into just another default preset or cultivation.


Update (Sept 2010): Vade and Tom Butterworth have developed plugins to use this effect in VDMX and Quartz composer. UCNV created a github page that is dedicated to datamoshing avi files and Eric Souther  made a MAX MSP patch.
These are just a few to choose from if you want to automate your datamosh effect!

To Smell and Taste Black Matter

To Smell and Taste Black Matter (1)


To Smell and Taste Black Matter (2)

Please watch the High Quality at Vimeo

Experiments with compression artefacts.
Two version of To Smell and Taste Black Matter - to show a part of the process (this step resulted in some black / dropped pixels).
(Synesthetic) musicvideo, created by exporting footage into different Compressions (such as least quality/animated/256 colors mov to least quality/256 colors/uncompressed avi, etc), resulting in artifacts such as dropped and bleeding pixels, noise and weird forms of interlacing.
The original footage is taken from a lofi webcam recording, in which Extraboy is performing an unfinished song...

GotoBirthday