Introduction: Haunted TV - Powered by Raspberry Pi & OpenCV

By Portfolio
Fusion Projects »
Happy Halloween!

This year I decided to expand on the Idea of a classic horror movie cliche: the mirror scare(a scene where a reflection is used to suddenly reveal a monster, ghost, etc.) I've used a Raspberry Pi, camera, and OpenCV to create a creepy camera feed on an old B&W TV which is triggered by face/eye detection. When the Raspberry Pi detects a face, it turns on the TV displaying a live camera feed. When the face looks away from the TV a ghost is displayed in the background(using background subtraction). Once the face looks back at the TV, sees the ghost and turns behind a second time, the ghost disappears and, after a few seconds the TV turns off, restarting the cycle.

I've also implemented eye detection to detect faces with masks, so you can use this as an outdoor decoration in 2020!

Outline

In the first step, I'll explain the inspiration behind this project, later I will give you a list of all the things you need to build this project on your own. Then I'll show you how to set up your Raspberry Pi to run this program. later I will show you how the face/eye detection and background removal algorithms work. After giving you a step by step assembly guide I'll end the Instructable with a troubleshooting guide.

Let's get started!

Step 1: Inspiration

As I mentioned above, the inspiration for this project came from a famous trick used in horror movies. I wanted to build a project that could create the same effect. The trick is usually done in 4 parts:

  1. A character goes to get something from the mirror cabinet, we see the character's reflection in front of the mirror.
  2. They open the cabinet or get distracted while doing something else.
  3. They look back at the mirror again and see something is behind them.
  4. They look behind and see nothing.

Here's a compilation of mirror scares from various movies to give you an idea:

I wanted to create this effect on a different medium, and I decided to use a crappy old TV. The low resolution and interference created a unique effect as if the TV was mirroring a parallel but different reality. This is where I came up with this Idea.

Step 2: Tools & Parts

All the necessary files (3D print files and Code) are attached to this step, also you can find the latest version of the code at this link.

Tools:

  • 3D printer (optional if you can use wood, acrylic, etc. to mount the Raspberry Pi)
  • Drill/Dremel
  • Soldering Iron

Parts:

3D Printed Parts:

  • Raspi Mount
  • Camera mount

Attachments

Step 3: Setting Up the Raspberry Pi

Note: You can skip this step if you have already installed the Raspberry Pi OS

Installing Raspberry Pi OS

note: you'll need a keyboard, mouse, and monitor to set up your raspberry pi

Before we get into the program, we first need to set up our Raspberry Pi.You can follow this official guide on how to install an operating system on your Pi:

https://www.raspberrypi.org/documentation/installation/installing-images/README.md

Make sure you install "Raspberry Pi OS (32-bit) with desktop and recommended software", this version will have all the software we need to run our program.

After you've installed the operating system and started your raspberry pi, open up the terminal and type:

sudo raspi-config

This will open up the configuration menu, navigate to Interfacing Options, and enable the camera and SSH.

Enabling SSH will help us get into the Raspberry Pi remotely so we won't have to use a keyboard, mouse, and monitor every time.

Now our Raspberry Pi is ready for our code!, in the next step, we'll assembly the camera and electronics to our Raspberry pi and test it with the Code.

Step 4: Assembly - Electronics

note: At this stage, you don't have to use a soldering iron. I strongly recommend you to use jumper cables, so you can change any component if it isn't working properly.

Raspberry Pi GPIO Pins

I've used the physical pin numbering as shown in the diagram below

Raspberry GPIO diagram

source: https://www.raspberrypi.org/documentation/usage/gpio/

The Camera

The camera port is between the HDMI output and Audio jack, insert the camera cable in a way that the open contact points on the cable face the HDMI port.

Connect the Receiver, RGB and Relay with jumper cables to your Raspberry Pi as shown:

YK04 Receiver

YK04Raspberry Pi 3
GNDGND
POWER3.3V
D012
D116

RGB LED

LEDRaspberry Pi 3
GNDGND
RED22
GREEN24
BLUE26

Relay

RelayRaspberry Pi 3
GNDGND
POWER5V
Data32

Step 5: The Code

We'll be using OpenCV(Open Source Computer Vision Library) for face detection and background removal. Let's install this library on our Pi,

Install OpenCV

The OpenCV Python module can be downloaded from the Raspbian repository, to do this simply open up a Terminal and type;

sudo apt update
sudo apt install python3-opencv

this command will install all packages necessary to run OpenCV.

reboot your Pi after the installation with;

sudo reboot

Haunted TV Code

First, navigate to the Documents folder;

cd Documents

next, install the Haunted TV code from the Github Repository by:

git clone https://github.com/gocivici/haunted-tv.git

This command will install the most recent version of the code.

After navigating into the project folder;

cd haunted-tv

you should be able to run the program by:

python3 hauntedTV.py

Now to test and debug the code you can use the keyboard instead of the remote. Use 'h' to take a haunted picture and 'a' to start face detection.

Note: If you get an error when you run the code, check the troubleshooting guide at the end.

Autostart

for more info and troubleshooting check: https://learn.sparkfun.com/tutorials/how-to-run-a-raspberry-pi-program-on-startup#method-2-autostart

We want to run our script automatically when the Raspberry Pi boots up. To do that we will be using a method called Autostart. With this method, the graphical interface will initialize first and then run our script.

Open the Terminal and type:

mkdir /home/pi/.config/autostart
nano /home/pi/.config/autostart/hauntedTV.desktop

Now copy the contents below inside the file you've just created:

[Desktop Entry]
Type=Application
Name=HauntedTV
Exec=/usr/bin/python3 /home/pi/Documents/haunted-tv/hauntedTV.py

save, exit with ctrl + x and reboot:

sudo reboot

Customization

To edit the code while in the project directory type:

sudo nano hauntedTV.py

There are three timers that are important for the jumpscare effect to work, these are:

timeTV = 5
timeGhost = 25
timeNoGhost = 10

These values are not in seconds. They are a value that is set by the faceFreqCounter. This counter starts when a face is detected, and continues to count if the face is still in the picture. If the face leaves the screen for more than 4 seconds the counter is set to 0. (This way false positives are filtered)

  1. timeTV: When the faceFreqCounter hits this value, the TV will turn on.
  2. timeGhost: When the faceFreqCounter hits this value, the ghost will be displayed after the face looks away. You can set this to a larger value if you want your guests to inspect the TV a little more before the ghost is displayed.
  3. timeNoGhost: This value determines when to disable the visibility of the ghost after the face sees the ghost and looks away.

You should also customize the background sensitivity in the code.

backgroundSensitivity = 20

This value determines the sensitivity of the background subtraction. You should increase this value If the haunted picture is not visible in the background.

You're done! Now it's time to test the setup with your TV.

Step 6: Face/Eye Detection & Background Removal (Optional)

In this step, I will get into detail on how I used OpenCV to detect faces and remove the background. You can skip this step if you just want to download and run the code.

Face/Eye detection with OpenCV

There are many methods to detect a face, the method I used is called object detection with Haar Cascades. Haar Cascade is a machine learning object detection algorithm that can detect objects in an image based on the concept of features. (Click here to learn how it works)

This method can be used to detect any object not just faces. In order to detect a face, first, we need cascade files. You can use any search engine to find various cascades of things you want to detect.

In this project, we'll use Face Cascade and Eye Cascade, you can find them here and here.

After downloading the cascades as .xml files we import them to our code:

import numpy as np
import cv2

#importing the cascades
detected = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
#detected = cv2.CascadeClassifier('haarcascade_eye.xml')

I'm just importing the face cascade for now, as I mentioned in the last step you can customize the code to detect faces, eyes or both.

Next, we start the camera and initialize a while loop so the code checks if a face is detected:

cam = cv2.VideoCapture(0)
while True:
    ret, img = cam.read()
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    faces = detected.detectMultiScale(gray, 1.3, 5)<br>

The cascade classifier, "detectMultiScale" simply detects the faces. Now we create a loop to draw a rectangle around the detected faces.

for (x,y,w,h) in detected:
    cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
    #put code here to run when a face is detected

finally we display the final image in a window:

cv2.imshow('img',img)

here's the full code for face detection:

import numpy as np
import cv2

#importing the cascades
detected = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
#detected = cv2.CascadeClassifier('haarcascade_eye.xml')
cam = cv2.VideoCapture(0)
while True:
    ret, img = cam.read()
    gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    faces = detected.detectMultiScale(gray, 1.3, 5)
    for (x,y,w,h) in detected:
        cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
        #put code here to run when a face is detected
    cv2.imshow('img',img)
cam.release()
cv2.destroyAllWindows()

save this file as FaceDetection.py and run it, you should be able to see a blue rectangle around your face.

Background Removal with OpenCV

I wanted to use a background removal algorithm to remove and replace the background with a haunted picture attached to the background. With this method, I was able to put the ghost behind the person. This was an important feature since I wanted to create the effect that there was a ghost behind the person. While searching the web on how to do this I found a very good tutorial:

In this tutorial, Misbah shows us how to replace the background with a video. however, to use this with our code we need to edit this for our needs. We want to display a picture instead of a video in the background when triggered.

There are also other background subtraction methods that you can use. I will link all the sources at the bottom of this step.

References:

https://pythonprogramming.net/haar-cascade-face-eye-detection-python-opencv-tutorial/

https://docs.opencv.org/3.4/d1/dc5/tutorial_background_subtraction.html

https://github.com/misbah4064/backgroundRemoval

Step 7: First Prototype & Test

For the first prototype, I built an adjustable stand for the camera and put it on the TV with the Raspberry Pi. At this stage, the main objective is to test the setup and TV to check if they're working properly.

  • Connect your Raspberry Pi to the TV with the composite cable (If your TV doesn't have a composite input use the AV-RF converter)
  • Connect the Raspberry Pi to power and wait for it to boot up.
  • Check if there is a problem with the video feed

Step 8: Design & 3D Print

There are two 3d printed parts for this build, one is the Raspberry Pi mount that will be glued inside the TV. The other part is the camera mount, which will help us mount the camera.

Note: You can edit/customize these parts in fusion360 they are attached to this step

The printer I used is the TEVO Tornado with the following settings:

  • Nozzle: 0.4mm
  • infill: %30
  • Filament: PLA

Attachments

Step 9: Soldering

There are two parts to solder. The LED and the antenna for the Receiver.

Step 10: Disassemble & Clean the TV

Safety First!

Even though we will be only disassembling a small part of the TV, it can be very dangerous if you don't practice safety procedures. High voltage may remain stored inside the tube even after being unplugged.

I highly recommend you to read MrJentis's instructable on this topic:

https://www.instructables.com/How-to-dismantle-a-CRT-monitor/

Step 11: Mounting the Raspberry Pi

Step 12: Mounting the Camera

  • I removed the headphone output jack on the TV and mounted the camera there.

Step 13: Final Assembly & Test

Now with your Raspberry Pi mounted to the TV, power up the setup. The Raspberry Pi should boot and run the Python script at startup, after a few seconds the TV should turn on, and you should be able to see yourself on the TV.

Next, find a spooky costume, get in front of the TV with your Remote and take the Haunted Picture by pressing B on the Remote. Now the script is ready to run. Press the D button on the remote, the TV should turn off and start detecting faces.

Test the setup by getting in front of the TV and look away after a few seconds. This should trigger the setup and now you should be able to see the Haunted Image in the background, if not, go to the Troubleshooting guide at step 13.

Step 14: Troubleshooting

+The Haunted picture is not visible.

-light is very important for background removal. Make sure the TV is in a well-lit area. Also move the light source behind the TV.

+I'm not sure if face detection is working or not.

-Go to step 6 and run the face detection code alone. You should be able to see a rectangle around your face.

+The CRT TV isn't displaying the picture.

-Go into the config.txt file in your raspberry pi, and make sure that the TV region is the same as your TV.

Leave a comment If you're having other problems. I'll collect them and try to answer them here.

+There's a lot of lag/latency on the live feed.

- I've used a Raspberry Pi 3 for this project. Any Raspberry pi with a lower processing power could cause latency and lag. But there are some optimizations you can do to prevent that.

Step 15: What's Next

In this step, I will list some ideas that I had. But did not find the time/resources to add them:

  • A Bluetooth speaker to make a noise when the ghost appears behind.
  • An Infrared Camera and LEDs could make this work in the dark. Which creates a scarier vibe.
  • A special stand for the TV with spooky decorations.
  • A more powerful computer, like the Nvidia Jetson which is built for visual tasks. This would reduce the latency.

Feel free to add more ideas or ways to improve this project in the comments!