Fly a Mini-drone With Your Computer!

Intro: Fly a Mini-drone With Your Computer!

A few months ago, I watched a TED Talk where they setup an indoor arena and did some amazing things with drones. It got me thinking, and it inspired me to build something like that for myself – but on a much smaller and cheaper scale. In the TED Talk video they use an expensive real-time infrared motion tracking system made by companies like Vicon or Optitrack to measure the position of the drones, and then used a computer to calculate and send control signals to coordinate the drones.

At a high level, my setup works in a similar way, using a USB camera to track the position of a $15 toy drone, and then using the computer to send radio control signals through an Arduino and a wireless card. Excluding the computer, all the equipment cost about $85.

Watch the video above to see my computer flying the drone! The photo above shows my first setup. Read this Instructable to learn how to do it yourself!

Also check out MakeHardware.com for more details and links.

STEP 1: Get the Components

To get this working, you're going to need a reasonably fast PC and the following items:

1. ELP 2 Mega-pixel USB Camera

2. Arduino UNO R3 Board

3. Addicore nRF24L01+ Wireless AddiKit with Socket Adapter Boards and Jumper Wires

4. Cheerson CX-10 Mini RC Quadcopter

5. Cheerson CX-10 Propeller Blade Guard

The Addicore wireless card is used to send the same radio signals that the drone would receive from the joystick.

STEP 2: Setup the Drone

To help the camera recognize the drone, place four paper circles on the drone. Cut out circles from colored card stock. You'll want to choose a contrasting color from the color of your drone. Then, install the propeller guards on the Cheerson CX-10 drone, and use some double-sided tape to affix three circles to the propeller guard and one to the center of the drone. Arrange the three propeller guard circles as shown in the photo, with one circle to the front, one to the left, and one to the right. The side with the power switch should have no circle.

STEP 3: Setup Your Arduino and the Wireless Card

Connect the nRF24L01+ card and the adapter board, and then wire them to the Arduino Uno according to the table below:

Arduino Uno NRF24L01+ Socket Adapter Board Wire Color (in photo) GND GND Orange 5V VCC Red D5 (Digital 5) CE Yellow A1 (Analog 1) CSN Green D4 (Digital 4) SCK Blue D3 (Digital 3) MO (MOSI) Purple A0 (Analog 0) MI (MISO) Gray Not Used IRQ White

The Arduino code is available in this GitHub repository. Now plug the Arduino into your PC and load the nRF24_multipro.ino sketch onto the Arduino with the Arduino IDE version 1.6.7.

STEP 4: Setup the USB Camera

The camera needs to be mounted so that it is aimed downward at a white surface from a height of about 2 feet. I used two sheets of poster board as the white background on the table. In my first setup (shown in the top picture), I used a folding table with a piece particle board to hold the camera above the table.

The second picture shows my upgraded setup built with Plexiglas sides and the camera mounted on wooden crossbars. In the second setup, the camera is mounted in the middle, and two LED lights are mounted next to it to make the lighting conditions more consistent.

STEP 5: Setup Your Computer

As you might expect there is a fair amount of programming required to get this working.

You'll need to install the following software:

1. Python 2.7 - I recommend using the Spyder development environment from the Python(x,y) distribution.
2. PySerial package
3. OpenCV2

Check out my guide to installing Python and OpenCV for more details.

Now download the code from the pc-drone repository. It is currently only setup to work on Windows right now, but you should be able to work on Linux or MacOS with some minor changes.

The main program to run is fly_drone.py, run it from the command line with python ./fly_drone.py

You'll need to have the USB camera and Arduino connected to run this code, as specified above. Most likely it will not work the first time - you'll need to edit the code so that it sees the right video camera, and also communicates to the right COM port for your Arduino. Also, you may need to tweak your code for your specific circle colors and also your lighting conditions. This post describes the process of how the computer recognizes the circles and how to modify the color recognition thresholds.

STEP 6: Make It Fly!

If everything is running properly with the fly_drone.py script, then it is time to turn on your drone. You should see one of the lights on the Arduino Uno stop flashing and stay lit. This indicates the Arduino and drone are paired and now the computer can send signals to the drone.

Press 'w' on the keyboard to let the drone take off and hover. To make it land, press 's'. There are other sequences available as well to rotate or move the drone through a pattern. You can find the keyboard commands in the fly_drone.py code.

The above video shows the USB camera images that the computer uses to fly the drone. The small blue square in the video indicates the target location that the computer is trying to fly the drone to. The colored circles drawn around the purple circles on the drone indicate that the computer identified the position of each circle.

I hope you enjoyed my project. Please vote for this Instructable in the Make It Fly contest!

Comments

[MarkusP24]

Hey, really cool project ! I do want to build something similar. Do I have to use exactly this camera, or can it be any other USB Camera ?

[LouisH43]

Hey there, I know that this is an old post, but I am trying to get it to work, however every time I try and launch I get:

C:\Users\hdgam\Desktop\pc-drone-master\fly_drone.py

Traceback (most recent call last):

File "C:\Users\hdgam\Desktop\pc-drone-master\fly_drone.py", line 19, in <module>

import blob_detect as bd

File "C:\Users\hdgam\Desktop\pc-drone-master\blob_detect.py", line 169, in <module>

calfile=np.load('camera_cal_data_2016_03_25_15_23.npz')

File "C:\Python27\lib\site-packages\numpy\lib\npyio.py", line 372, in load

fid = open(file, "rb")

IOError: [Errno 2] No such file or directory: 'camera_cal_data_2016_03_25_15_23.npz'

I have checked to see if the 'camera_cal_data_2016_03_25_15_23.npz' is there, and it is... an ideas?

[DhruvS39]

I keep getting this error when I compile.

Arduino: 1.6.11 (Windows 10), Board: "Arduino/Genuino Uno"

C:\Users\Dhruv Suri\Documents\Gesture Quad\LeapCopter-Arduino\LeapCopter-Arduino.ino:59:28: fatal error: iface_nrf24l01.h: No such file or directory

#include "iface_nrf24l01.h"

^

compilation terminated.

exit status 1

Error compiling for board Arduino/Genuino Uno.

This report would have more information with

"Show verbose output during compilation"

option enabled in File -> Preferences.

Is there a way to fix this?

[CláudiaC84]

Hello. Does anyone know how to connect a mini drone mars the pc to program with scratch.

thanks

[Sverd Industries]

Amazing project! What do you think of running the whole thing on an intel edison? They can handle python and USB cameras, as well as arduino code. It also comes with buiilt in wifi so you won't need a wireless card.

Would be cool to see the whole thing on a small self contained unit!

[perrytsao]

It would be nice to be able to run it all from one self-contained unit. I am not sure if the Intel Edison is powerful enough to handle all the vision processing.

A separate wireless card would still be needed since the CX-10 drone doesn't use standard 802.11 wifi, it uses a proprietary chip compatible with the nRF24l01 card I used. The nRF24L01 card can interface with the Intel Edison easily enough though.

[theoriginalrage]

This is a really cool project. I'm wondering, could you bypass using the Python script and just send the commands via the Serial Port in the Arduino IDE?

[KrunalP19]

hi ,

i am having error running the code,i have attached an image file ,pls guide me on this

thanks

[perrytsao]

Hi KrunalP19,

I couldn't read your screenshot because it was too blurry. I'll PM you my email so we can discuss more.

Perry

[KrunalP19]

Hi,
Pls do pm me my I'd [e-mail]

[KrunalP19]

Hi,

Can i use sony ps3 eye webcam for motion detecting ?

Thanks

Anjan

[perrytsao]

Yes, I think it could work. The PS3 eyecam has good latency and FPS rate, but it's image quality and color is not that good. Some adjustments in the software may need to be made to accommodate those differences versus the ELP camera I used.

Check out my article where I compared the PS3 eyecam to some other USB cameras:

http://www.makehardware.com/2016/03/29/finding-a-low-latency-webcam/

[dick55]

Impressive project. There is a lot to learn here: using OpenCV2 to track the drone with camera input to a PC; communicating with the drone's receiver using a compatible transmitter and understanding the command format; and closed loop control dynamics. Not a simple undertaking.

[perrytsao]

True, there were a lot of things to bring together to get it to work, but a lot of the heavy lifting was done with open-source software. The computer vision code was made much easier by using the OpenCV2 library. The Arduino code to send signals to the drone receiver only required some minor changes to accept commands via the PC's serial interface. The only stuff I had to write from scratch was the closed-loop control which turned out to be four relatively simple independent PID loops. Anyone with the willingness to do some hacking in Python should be able to do it!

[agidalolade4]

Awesome,pls do u know where I can get an arduino.

[jdaniel2004]

RadioShack will usually have an arduino, if not then I would go to Amazon

[seamster]

So cool! :)