Slit-Scan Video Primer

Slit-scan video is a way of developing video where a single frame of the video includes information from multiple moments of time. Instead of all the pixels of the frame being from a single instance, the pixels of a single frame are sampled from different points in time. There are many ways of producing this effect (both digital and physical), but it’s characterized by strange distortions of movement – kind of like a fun house mirror. The intent of this Instructable is to give you a tool to help you visualize slit screen video, which will hopefully help you generate your own slit-scan video ideas, and understand how other slit-scan video techniques are used.

Step 1: Volumetric Interpretation of Video

To begin, first imagine that each pixel in a frame is a cube instead of a square. When we consider each frame, it now has a depth. This allows us to stack all the frame of the video, one on top of the other, to make a rectangular volume of video. All the information of the source video is represented in this volume. Notice that the axis along its length is time.

To illustrate this, I’m using three simple frames. Imagine that the colored pixels are a thrown ball, rising from bottom left to top right.

Now imagine giving a depth of one pixel length to the first frame.

After adding a depth to the other frames, they can be stacked to form a volume, with time along one axis.

Step 2: Slicing the Volume

Once the volume is formed, it can be resampled to form new frames. If we sliced up this volume along the time axis, we would recreate our original video. To create a slit-scan video, we only need to slice up this volume in new ways, in literally any other direction.

For example, I’m going to take slices entering from the side (perpendicular to the horizontal axis), instead of the front.

This will create new frames. Notice that now time is the horizontal axis of these new frames. Each new frame is one column of pixels from the source video, over the entire length of time.

Step 3: An Example

Here is an example using real footage. Below is the visualization of the volume to be resampled (only one segment of the video is represented):

Here is pseudo-code for this specific effect.

% Create volume from video frames

for f=1:totalFrames

currentFrame = video(f)

for m=1:rows

for n=1:columns

volume(m,n,f) = currentFrame(m,n)

end

end

end

% Now resample the video to form new frames

for n=1:columns

for m=1:rows

for f=1:totalFrames

newFrame(m,f) = volume(m,n,f)

end

end

newVideo(n) = newFrame

end

Step 4: Other Slices

Slices can be taken in other directions to produce other effects. Or, instead of a rectangular slice, the slice might have a shape to it, have a fold in it, or even be animated. Following are links to some projects who have used a slit-scan effect. See if you can figure out the shape of the slice they are using and how they are moving it through the volumetric space.

Camille Utterback: http://camilleutterback.com/projects/liquid-time-series/

Daniel Crooks: http://vimeo.com/77654682

Khronos Projector: http://www.youtube.com/watch?v=dZN2ICTRWBU