Interactive Multitouch Display

Between the Apple iPhone and Microsoft's interactive table, multi-touch displays are all the rage. This instructable will show you how to turn your lcd projector into an interactive multi-touch display table using a few cheap components readily available from the hardware store.

Here is a video of my display in action:

Step 1: Theory of Operation

This multitouch display screen design is based on the description in Jeff Han's paper,

Han, J. Y. 2005. Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology

The figure below comes from his web site.

An acrylic panel is edge lit with infrared leds. When your finger comes in contact with the acrylic, it scatters infrared light out the back where it is visible via infrared camera. As long as nothing is touching the acrylic, very little of the light escapes, instead just reflecting around inside. Image processing takes care of detecting tips of fingers and relaying their location to application software. Since the camera "reads" the whole display in parallel, it is easy to detect multiple fingertips at once, even those belonging to multiple users. All this sensing goes on in the infrared spectrum, leaving us free to utilize the visible spectrum to display interactive software.

Since most hobbyists can't afford multiple projectors (i don't even own one, just borrowed it from dr.eel), my design uses a ceiling mount that swivels so that the projector can be used either in standard mode (say for watching movies) or can be aimed downwards, bouncing off a reflector and onto the multitouch display screen.

The screen itself can be constructed from hardware store materials and hand tools. Excluding the projector and modified webcam (commodity items these days), the only thing complicated is the software. Halfway through this project, I was happy to discover that there is thriving DIY community which has already undertaken the task of writing the image processing code and several cool open source demos which can be found here:

touchlib

Step 2: Screen Frame

The primary component to be constructed is the screen itself. This is a piece of acrylic, with a frame which holds the diffuser and IR leds in position. I opted for a sandwich style construction out of 1x2s and aluminum channel. My acrylic was 30x36x0.25 inches so I made two frames to match, mitering the edges and assembling with screws and construction adhesive. The sandwich design is simple and leaves plenty of room for wiring the leds.

Step 3: Led Rails

The leds are mounted in extruded aluminum c-channel, pressed into holes. The channel serves to hold the leds in place as well as providing a baffle to direct the light into the edge of the acrylic. Additional spacer blocks cut from acrylic keep the panel from sliding around and pushing the leds out of their holes.

Step 4: Led Mounting and Wiring

I used 88 infrared leds I ordered surplus online. Their maximum output was 10mW @ 940 nm. Each led needed 100mA at 1.45V so i wired sets of 8 in series along with a 5 ohm resistor. I wired these sets in paralell across the 12V rail of an old computer power supply I had lying around. Since the total current draw is over an amp, you can't get by with a wall wart.

Pour yourself a glass of whiskey and fire up the soldering iron...it will take a while to wire up 88 leds.

Step 5: Prepare Acrylic

It is valuable to polish the edge of the acrylic to maximize the light that enters from the diodes. Some folks advocate drilling holes in the edge so the leds seat down in but they seem to work ok just butted up against the polished edge. Start with 200 grit sandpaper and work up to 600 or realms beyond. Sanding always takes longer than you think.

Step 6: Screen Sandwich Assembly

Once the frame, rails and acrylic are prepared you are ready for the final assembly. Clamp the layers together for easy drilling and bolt them in place. Once the sandwich is assembled, it is necessary to caulk along the edge of the aluminum in order to block any light that might spill out thru the crack. We want it all nicely ducting along the inside of the acrylic instead.

In order for the projector to display on the surface it is necessary to also introduce a diffuser (not shown here). A sheet of thin drafting paper was included in the sandwich supported by a second thin layer of plexiglass.

Step 7: Modify Webcam

In order to image the infrared light, we need an infrared camera. Fortunately CCDs in cheap consumer cameras are quite sensitive to IR, so sensitive in fact that it is necessary to filter out the ir in order to get good pictures. I got lucky and had an old intel web cam lying aroud in which the ir filter was a cinch to remove. Replace the ir filter with a visible light filter (so the camera doesn't see the projected display). I used the exposed end of a negative. See e.g., this instructable for more details.

Step 8: Projector Mount

A key aspect of my design is the projector mount. It allows dual use, whereby the projector can be used with a standard projection screen, or swiveled down for use with the mutitouch display. I cobbled something together using scrap metal from the closet. A central support and cross piece with holes drilled to match the mounting holes on the bottom of the projector are attached thru a bolt to the wall. The bolt provides a natural place for the projector to hinge from.

Step 9: Framing Support

Last but not least, build some framing to support the screen. I started from a coffe table frame as a base and a couple wood planks to get the screen up in the air. The exact geometry is dependent on the optics of your projector. The one I borrowed wouldn't focus at close distances so it was necessary to have the light follow a longer path, bouncing off the mirror and back up onto the display. The angle of the display surface matches that of the projector (angle of incidence = angle of reflectance) in order to minimize keystone distortion.

Step 10: Software and Demos

The key thing to making this all work is really the software that converts the detected blobs of light where your fingers are into awesomeness...of course a good soundtrack also makes a huge difference. I used touchlib which basically just worked out of the box. There is a calibration utility which saves results to a configuration file that is shared among the other apps. I also downloaded some Flash apps, including the light-box shown in the video. These interface with touchlib via OSC, a neat protocol for relaying timestamped events which should provide a good starting point for building your own apps.

Step 11: Enjoy

Excluding the projector, the total material cost is on the order of $100-200 depending on what you already have lying around. Construction time was around 15 hours, mostly soldering and sanding. There is a very active DIY/academic community which has started exploring this space; lots of reward and interesting things to be done for little work with very simple materials

touchlib
nui group
multitouch blog

Enjoy.