Introduction: Poor Man's Google Glass/Aid for Those With Tunnel Vision

By plus.google.com/1131131141217269
About: Currently attending UCSD. L'Chaim.

Abstract:

This project streams live video from a fish-eye camera onto a wearable heads-up display. The result is a wider field of view within a smaller area (the display is comparable to a 4" screen 12" away from your eye and outputs at 720p). The primary purpose is to provide more spacial awareness to those with Retinitis Pigmentosa (abbreviated as RP), a degenerative eye disease that results in tunnel vision (it is comparable to looking through a toilet paper roll all the time). That being said, because the project is based off an RPi, there is a huge range of possible applications (some of which will be discussed at the end).

Materials List:

Total Cost: ~265 USD (depends on what you already have and how much you pay for shipping)

Tools List:

  • Wire Cutters
  • Wire Strippers (for wire sizes of about 26 awg)
  • Hot-Glue Gun (and glue stick for said hot-glue gun)
  • Keyboard
  • Screen (must be HDMI capable, but again, with good enough eyesight, Vufine can be used as a monitor)
  • Soldering Iron (and solder)
  • X-Acto Knife

Personal Note:

This project was inspired by my grandmother. She was diagnosed with RP a long time ago and her eyesight has been deteriorating over the years. Recently, she decided to give up her seeing-eye dog, Ventura (nicknamed Seymour, pun intended), because of age. Though this may have made life easier in some aspects, I'm sure it has made other things more trivial (such as walking through a mall). I had already been looking into making a device like this for a while, but there were no readily available (reasonably priced) screens. That changed when I found Vufine. It's a great company with a great product, and no, I'm not being paid to say this; it's a genuinely good company. As for my grandmother, she has reported that this device has just about doubled her field of view (in terms of diameter, not area).

Update:

I built this with a Vufine module configured for the left eye (my grandmother is completely blind in her right eye), so there may be some issues for those building this for the right eye (nothing big, just a change of orientation for a few parts). That being said, Vufine has been kind enough to send me a module for the right eye so I can continue this project and fix those issues. As soon as I do, I will further update this instructable.

Step 1: Setting Up the Raspberry Pi

Setting up the Pi's Hardware:

Note: If you have a Raspberry Pi other than an RPi Zero (W or regular), you can make the necessary software edits using that Pi and transfer the SD card to the Pi Zero.

Second note: I didn't actually buy some of the hardware that connects to the Pi; I have a convenient hat instead. That being said, the picture above is what your set up should look like. It is not mine and was taken from here.

First, we will plug in the keyboard. Since your keyboard has a type A USB connector (the ubiquitous rectangular one) and the Pi Zero doesn't, we will have to use the USB adapter. Plug the USB adapter into the USB port closest to the middle of the board. Then plug the keyboard into the USB adapter. The other USB port will be used for power. The top two cables in the picture are USB cables with the very top cable providing power.

Second, plug the HDMI cable into the mini HDMI port on the Pi and the HDMI port on your screen (if you chose to use the Vufine display as your screen, use the HDMI ribbon cable).

Setting up the Pi's Software:

This process is documented everywhere, but the official guide is here. It is important to note several things: you will be using the micro SD card that you bought (the one I chose has an adapter so you can flash it with a regular computer), you will be installing Raspbian, and you should enable the camera in the Raspberry Pi configuration menu. Additionally, DO NOT follow the tutorial in executing "startx". If you did, no biggy, but you will need a mouse now. Go to the command line (should look like a black window in your taskbar) and use that (the "pi@raspberry ~ $" should appear within that window). Now we will turn off the Raspberry Pi by executing:

sudo shutdown -h now

Step 2: Installing the Camera

Physical Connections:

First, let me explain how the ribbon cable connections work. To connect and disconnect ribbon cables, one must first lift the tabs on the side (for the Pi Zero, this is an outward direction), which will allow the connector to fold in and out to some degree, thus allowing the ribbon cable to come in and out. To secure the ribbon cable in place (and thus complete the connection process), you must press the folding piece of the connector flat against the cable and reinsert the tabs. The cable should now be secured within the connector. To see an official video on how this is done, click here.

It is important to note that the white cable that comes with the camera will not fit for the Pi Zero; make sure to replace it with one of the golden ribbon cables. Additionally, make sure the exposed side of the cable is facing downward, into the Pi and the camera boards.

Software:

The camera's setup is relatively easy on the software side. To stream video directly from the camera to the HDMI, simply execute the following line in terminal:

raspivid -t 0 -rot 0

The camera should now be streaming to your screen. Now, stop the stream by pressing ctrl+c. Now we will be executing the same command, but if you bought a standard Vufine module (configured for the right eye) change the 0 after "-rot" to 90. Likewise, if you bought a Vufine display configured for the left eye change the 0 after "-rot" to 270.

Now, if you hold the camera against the Vufine display in a manner similar to the way it's shown in the photograph, you should see an upright image (on your screen, not necessarily in the Vufine). If your image is upside down, try the other number (270 if you did 90 before and 90 if you did 270 before; you still need to press ctrl+c to get out of the stream).

Once you have the camera's stream correctly oriented (via software), we'll want to make sure it starts whenever the Raspberry Pi is turned on. The first step is to execute:

sudo nano /etc/rc.local

Now, type out a command similar to what you executed earlier ("/usr/bin/raspivid -t 0 -rot 90/270") in the lines between "fi" and "exit 0"(keep it on one line as seen in the picture).If you would like a lower resolution image with a higher frame rate, use "/usr/bin/raspivid -t 0 -rot 90/270 -md 6" instead. After typing this out, you should save it by pressing ctrl+x, entering y (yes, you would like to save it), and then pressing enter to save it under the same name. You should now be back at the command line ("pi@raspberry ~ $"). Finally, you can test to make sure everything works by rebooting the Raspberry Pi with:

sudo reboot

If everything works, you should see the camera streaming video to the HDMI output at the end of the boot process.

Notes:

  • After all of the above has been completed, you may want to consider applying a small amount of hot glue to the ribbon cable/connector on both the camera and the Pi to make sure it stays in place. This is purely optional and is something I haven't actually done.
  • If you have a Vufine display configured for the right eye and you choose to orient it differently (it's more convenient if the Pi's USB ports are on the bottom), the camera's top side will be on the left when you're wearing it.
  • The camera in the picture is already mounted. We'll get to that in a later step.
  • I wanted to post a picture of the camera connector on the Pi, but I don't want to take the Pi off if I can't put it back on and I lost all of my zip ties (which you'll see are necessary for mounting it in a later step) and I'm not sure when I'll next be able to go buy some more, so I've decided to just upload now and update later.

Step 3: Charging the Vufine Display

The Vufine display has a battery of its own, but it lasts only 90 minutes, and while this may be enough for the Vufine's intended usage, it's insufficient for our purposes. That being said, we can charge the Vufine while it's being used, and thus extend its battery life. However, regular USB cables are rather large and since we want to minimize the weight, we will have to modify one (making it smaller and lighter in the process); we will strip a USB cable down to the very minimum.

Note: The rest of this step details how to get the connector from a USB wire. That being said, I was made aware of these cheap (in cost) USB connectors. Using them would allow you to skip the dangerous part of removing the connector from the wire. I have no experience with them, but I'll update this instructable on how to use them once I do.

Preparing the USB Cable

Note: This is, by far, the most dangerous part of the build. Extreme caution should be exercised around sharp knives

The first step in "minimizing" the USB cable is to cut it about 5 inches (I would say about 7 for a right eye configuration) up from the micro USB end. Continue by stripping the wire all the way down to the USB micro end.

Now, for the dangerous part, you'll have to cut the plastic encasing off using the X-Acto knife. I suggest using a set of pliers or a table vice to hold the plug while you cut it to minimize the risk of cutting yourself. The best way to cut it off is to cut along the seams on the side (they look like lines) as seen in the picture. You should be using a decent amount of force to dig the knife into the casing. After a while, one side may be able to "swivel" outward, and depending on how free it is, you may choose to use a pair of pliers to rip the rest off rather than cut it off.

Finalizing the USB Piece

Once you are left with just the metal piece, you should remove the two middle wires (the non red and black ones). You can do this by either cutting them off or using a soldering iron to desolder their connections and simply take them off (I suggest desoldering if you have enough experience). Next, put some hot glue on the base of the wires (where they meet the Micro USB plug) to provide an insulator and to keep it secure. When you're done, the image should look something like the USB in the picture (we haven't connected it to the Pi yet, though).

Connecting it to the Pi

This step is rather simple. All you have to do is solder the red wire to the 5v port and the black wire to the ground (GND) pin on the RPi. To see exactly what ports, see the image with the red rectangle on the Pi. To see a video on basic soldering, click here. I suggest you use some hot glue on both sides of the Pi (where you soldered) to secure the wires and to provide an insulator.

Step 4: Connecting It All.

Securing the Pi

At this point, place the Pi against the Vufine module with the camera port near the front and the USB connectors (installed on the Pi) facing downwards and inward. Secure it using the two pairs (one pair for each side) of zip ties. Do this by putting a zip tie through one of the top corner holes on the pie, wrapping it around the Vufine (downward), and putting it through the bottom corner hole on the same side of the Pi. Then take another zip tie and lock it on to the first. Tighten it so the Pi is pulled close to the Vufine. Cut off the remaining zip tie body for both zip ties (all for the second, and the part that sticks out of the second zip tie for the first). Repeat this step on the other side of the Pi.

Plugging Stuff in

This is perhaps the simplest step of the entire build. Plug the USB cable you soldered onto the Pi into the Vufine display.

Hot Gluing the Connector

The magnet in the Magnetic Docking Station isn't really strong enough for our purposes, so we'll add some hot glue to strengthen the connection. Attach the Vufine module to the Magnetic Docking Station it comes with (not plus) with the strap up (as shown in the picture). Make sure it's just about parallel with the actual Vufine (you'll still be able to adjust the display's position by moving the strap around on the glasses). Apply a decent amount of hot glue on both the top and bottom. Let it cool before moving on to make sure it doesn't get jostled into a weird position before it solidifies.

Plugging Stuff in, Continued

Though this part is not quite as easy as the first Plugging Stuff in part, you shouldn't struggle too much. Simply plug the HDMI ribbon cable into both the Vufine and the Pi (the struggle is more psychological than physical; you can do it. I believe in you).

Connecting the Camera

Left Eye Configuration:

This is an easy part, but the camera's positioning makes it hard to describe with words. Essentially, you'll be attaching the camera so that it's sideways with the ribbon cable looping under it. See the picture to see what I mean. Press it against the Vufine screen and secure it in this position by applying hot glue to both the top and bottom. I wouldn't put any on the sides as doing it on just the top and bottom is sufficient for securing it and doing it on the sides might prevent any sideways adjusting of both the ribbon and the Vufine screen.

Right Eye Configuration:

I haven't done the right eye configuration before (will come back and edit once I do), but essentially you need to make sure that the camera is sideways and in a reasonable position on top of the screen. The top (the side opposite the connector) should be facing the left (left when you're wearing it). Use hot glue to attach the camera to the screen piece on the Vufine module. Additionally, you can use hot glue (or zip ties) to make sure that any excess ribbon cable is secured (give it some lee-way for adjustability and try not to crease the ribbon as it is not intended to withstand high degrees of rotation). You can also go back to Step 2: Installing the Camera if you have to reconfigure the camera stream as a result of a change in its physical positioning.

Step 5: Fin.

Turning it on

To turn it on, simply turn the Vufine module on by pressing the gray button on the back. It should start flashing blue. Next, plug in the Pi Zero using the far right USB port (on the outside and labeled PWR IN). You should see now see a solid red light on in addition to the flashing blue LED on the Vufine module. This means that the Vufine is being charged. Additionally, the screen should start and you should see the Pi go through its boot process with the camera stream coming up at the end. Once the screen comes up, the blinking blue light on the Vufine button should become solid.

Note: If you turn the Pi on first (followed by the Vufine), your screen might come out weird. To fix this, turn the Pi off and then turn it back on.

Turning it off

Turn off the RPi by unplugging the USB that provides it with power. To turn off the Vufine, you can either let it turn off by itself (which it will do after not receiving a signal for a short period of time) or you can hold down the grey button (with a blue LED) on the Vufine until the blue LED turns off. I prefer letting it turn itself off, but you do you.

Further Improvements

As said in the abstract, because the project is based off a Raspberry Pi, the project has a huge ability to evolve. For starters, here are some ideas I had:

  • You could hook it up to a wireless keyboard for convenient use as a heads-up linux computer. I would use a keyboard (and mouse) like this and I would hook it up in a process similar to this.
    • Adding a hand-motion based interface like this would also be cool.
  • Add buttons to the side for saving pictures and videos and for zooming in and out
  • Add a native power switch (I say native because some USB cables have switches built in)
  • Make a 3D printed case for everything.
  • Add an audio output
  • Add a battery attachment
    • Putting it on the other side of the glasses would be a convenient way to add a counterbalance (weight on the glasses is currently a little lopsided).
  • Make a CNN (convolutional neural network) with TTS (text-to-speech) output for an audio description of one's environment. This neural network could run on a Movidius Neural Compute Stick for more computational power.

You can see some of these ideas being implemented on my github page.

Note:

I'll probably add some more pictures later. If you want any picture in particular, please let me know and I'll do my best. The same applies for explanations; if you want me to go into something in more detail let me know and I'll try to get it done.

Thank You

  • I learned a lot from this community; hopefully this helps others as so many here have helped me before.
  • Again, this is my first instructable and any and all feedback would be much appreciated. If anyone does do something with this, please let me know; I'd love to see where the community takes this.

Special thanks to my grandmother for everything she's done for me throughout my life. I'll never be able to pay you back, but I'll be sure to try my best.