Introduction: Android Powered Robot (with FPV, Wi-fi / 4G Control )

Since my school days (now I am a college student) I always wanted to make a RC car with live video feed, Wi-Fi or internet control features. I started with a raspberry pi 3 B+ board but it requires other components like Camera module, 4g LTE shield etc. which are expensive so I changed my mind to use my smartphone (has cameras, IMUs, compass, GPS, etc..) as a controller to control the Arduino UNO board (controls the motor).

Using an android phone has more advantages like we don't need to write drivers for modules. We can just use the android SDK to access the components like cameras, microphones, USB, 4g, Wi-Fi, Bluetooth etc.. In this instructable shared how my project works and also how you can make your own version of this project.

Supplies

  1. Arduino UNO R3
  2. L293D Motor Shield Battery
  3. 4 x BO motor
  4. Android Phone (2 if using android remote)(version >= Android 8.0)
  5. PC
  6. Prebuilt or DIY robot chassis
  7. Li-Po Battery
  8. USB OTG cable
  9. Arduino USB cable

Step 1: The Platform (4WD) :

My version of the platform was made of 3 wooden reapers. In the center of the platform a mobile phone holder is mounted to hold the phone in a landscape form. 4 BO motors are mounted on sides using the mounting clamp. The motors are controlled by a L293D Motor Shield (stacked on Arduino UNO). I used a 3 cell Li-Po battery as a power source.

If you are making a 4WD car make sure the distance between each motor is the same (like each motor is placed on each corner of a square).

You can also use your own chassis after changing a few lines of code.

Step 2: ​How It Works (Overview) :

This project consists of 3 software components the TCP Server (for pc), an Android TCP Client app (controls the Arduino), an Android TCP server app (for mobile)

The Android TCP Client app is written in Java that controls the Arduino board for low level hardware components like driving the motor shield.

The TCP server (for pc) is written in python and uses PyQt5 framework for GUI. This software contains widgets to control the TCP client app and to visualize telemetry data from the TCP client app.

The Android TCP server app (for mobile) also works same as PC version but limited to Camera feed viewer and a Joystick to control the RC car.

TCP stands for Transmission Control Protocol a communications standard that enables application programs and computing devices to exchange messages over a network. It is designed to send packets across the internet and ensure the successful delivery of data and messages over networks.

Dash source : https://github.com/redLoneWolf/Dash-2.0

Craze app (client) source : https://github.com/redLoneWolf/Craze

CrazeRemote (server) source : https://github.com/redLoneWolf/CrazeRemote

Step 3: How It Works (PC TCP Server / Android TCP Server) :

Once we start the TCP server it listens for any incoming connections in a background process. After receiving a request from a client, the server asks the client type (MAIN, CAM, DEPTH_CAM) of the client.

Data and Commands are exchanged between the client and server using the data frame format shown below.

| Preamble($) | Size of payload (n) | Command | Payload of Variable Size (n) |
|   1 Byte    |     4 Bytes         |  1 Byte |     n Bytes                  |

The preamble $ sign marks the beginning of a data frame, Size block contains the size of payload (if payload is needed else 0), Command block contains the byte value of command (ex. START_TELEMETRY = 101), payload block contains the data to be transmitted.

If data is available to read in the socket then the available data are appended in a buffer And checks the first byte and continues to process if the first byte is the preamble else discards the data in the buffer. Payload in the received data frame is processed according to the command in the command block.

The android TCP server works the same as pc server but written in java. It contains Camera Feed viewer and controls.

Step 4: How It Works (Android TCP Client) :

After connecting to the server using the IP address, it sends the client type to the server.

The client maintains the connection in a background service. Data and commands are received similarly as on the server.

The android phone and the Arduino UNO (Microcontroller) are connected together using USB OTG cable to provide serial connection. If the command is to execute some task in the Arduino it sends the command and data to the Arduino over USB. The Arduino contains instructions to execute for the corresponding command.

The app also contains the TfLite Dense Depth ( thanks to https://github.com/jojo13572001/DenseDepth ) that can generate depth images from a monocular image. Later we can use that depth image to provide autonomy or obstacle avoidance features.

Step 5: How It Works (Arduino Part) :

The Arduino and android app communicates through USB serial once the connection is established the app sends a handshake signal if not done handshake before, then only the command and data transfer takes place.

The Arduino code accepts 4 motor speed values from the app via serial and applies it to the corresponding motors. The direction of motor rotation is based on the sign of speed value received from the app. That is If the speed value is positive then the motor is made to run in forward direction and If the speed value is negative then the motor is made to run in backward direction.

Step 6: How to Use (PC Server) :

1. Clone or download the Arduino code https://github.com/redLoneWolf/Android-Controlled-Robot

2.Now build and upload the Arduino code to the Arduino board.

3. Clone or download the Dash repository https://github.com/redLoneWolf/Dash-2.0

4. Open CMD inside the Dash-2.0 directory and type command and hit enter this will install required python libraries inside a virtual environment.

pipenv install -r requirements.txt

5. After installation activate pipenv by executing 

pipenv shell

6. Now run the main_2.0.py file to open the Dashboard by executing the command

python main_2.0.py

7. Then install the Craze app (client) on the android phone. After installation connect the Arduino and phone using USB cable.

8. On the dashboard select how you want to make connection, local or ngrok(requires auth token from ngrok account) and click the Go to Dashboard button

9. Click the Start server button to start the TCP server.

10. After the server starts, enter the displayed IP address/Hostname and port number on the craze app and click connect.

11. In the sidebar click the Connect USB button to connect android and Arduino.

12. Click the start camera feed button to start live video feed from the phone.

13. Click the Start RC button to enable WASD key inputs from the keyboard.

14. You can press W to go forward, S to go backward, A to go left and D to go right.

15. The start Depth Feed button starts/stops the depth feed from the phone.(takes a few seconds to load the Dense Depth model)

16. The start telemetry button starts/stops the transmission of telemetry data(acc, gyro, mag, battery, heading).

17. To stop the server click the stop server button.

Step 7: How to Use (Android Server) :

  1. Download the CrazeRemote App APK file and install it on another android phone.
  2. Open the app and select how you want to make a connection, local or ngrok (requires auth token from ngrok account). If you want to use ngrok long press the ngrok checkbox and enter your ngrok auth token that appears on the screen and click ok.
  3. After selecting the connection type start the server by clicking the start button.
  4. After the server starts, enter the displayed IP address/Hostname and port number on the craze app and click connect.
  5. Click the Connect USB button to connect android(craze app) and Arduino.
  6. Click the Start Cam button to start live video feed from the phone.
  7. Now you can use the joystick on the screen to control the robot movement.

  8. To stop the server click the stop server button.

Step 8: Test Videos

Test Using PC Server :

Test Using Android Server :

Feel free to edit the code.

Thank You.