Introduction: Training Buddies: Making Physical Excercise Enjoyable for Children
Demo Video Training Buddies (click to start the video)
In this guide we will detail how we created an smart exercise agent or robot, for the Intel IoT Roadshow Amsterdam 2016. Our idea was:
Training Buddies engages kids in physical exercise in a playful way. A buddy will encourage the child to perform a series of physical tasks, and will be rewarded in the form of spoken compliments. The exercises can be prescribed by a physiotherapist, and the buddy makes sure the exercises are done.
The buddies are connected to a Cloud server using their build-in WiFi, where the parents and therapists will be able to monitor how the kids are doing, and see the status of the Buddies, as well as updating the tasks to perform. Built in Bluetooth Low Energy from the Intel Edison board is used to detect nearby buddies, owned by other kids, so kids can play together.
This guide explains how to build a comparable near-sentient agent, built on the Intel Edison board with an Arduino breakout board, and a Grove Basic Shield V2, with basic sensors from the Grove starter kit.
The code is written in NodeJS and can be found on GitHub:
Step 1: Constructing the Buddy
Parts used for a single Buddy (2 made in this example):
- Small cardboard box 20x15x15cm
- Coloured paper to "dress" Buddy
- 4 cards with facial expressions to display a mood
- 2 paper cups as arms
- 5x5cm piece of cardboard to form the base of the head
- A prop to make the motor connect with the base of the head (image 3)
Angle the box on its side, so that the opening is in the back.
Cut 2 holes on the sides of the cardboard box, so that the paper cups can fit through. Make sure you do not cut this hole in the opening of the box. (Image 1)
Cut enough coloured paper to cover the front and sides of the Buddy, and glue to the box. (Image 2)
Glue the face cards on the flat cardboard piece to form a square. (Image 2)
Cut a small hole in the center of the top side of the box, just big enough for the motor rod to fit through, and attach the head (Image 3)
Place the Intel Edison board inside the box.
Step 2: Wiring the Board
Parts for ONE buddy (2 made in this example):
- Intel Edison Board
- Intel Edison Arduino Breakout Board
- Grove Basic Shield V2
- 2 Touch sensors
- Stepper motor, and Stepper motor driver
- 3 Axis accelerometer
- Piezo vibration/bend sensor
The connected peripherals are the following:
Analog 0: A Grove Piezo Vibration Sensor, to detect the high five on the arms of the Buddy.
Digital 2: Grove Touch Sensor, so touching of the left hand is detected.
Digital 6: Grove Touch Sensor, touching of the right hand.
I2C: A Grove 3 axis accelerometer v1.3 is connected, and besides the normal connection you need to also wire the interrupt pin, as shown in image 1. This allows us to offload some of the shake/lift detection to the accelerometer itself, simplifying the NodeJS logic. The interrupt pin does not have nice headers so you need either a (slightly) thicker cable or you need to solder a cable to the interrupt. We connected the other end of the interrupt to GPIO pin 4.
GPIO: Stepper motor driver plus stepper motor, in order to turn the head, depending on the mood of the Buddy.
Step 3: Code the Client
The client is coded in NodeJS. If you want to skip this part, and just build the exact same demo routine as we did you can download the code on the GitHub repository: https://github.com/hermanbanken/thomas-bot. When building your own logic this repository might also contain handy examples.
Architecture
Since the server will be written in Meteor, to speed up the development, the Edison needs to connect to the DDP api of the server. DDP is like a REST api, but is JSON only, and updates reactively on the client when the data on the server changes.
The entry point of the client is main.js. There we will first authenticate to the server and then execute some routine or wait for instructions. If you copied the code, you will need to adjust the hostname or IP of the server.
Several separate functions are referenced in main.js: the logic for the accelerometer (accel.js), the touch sensor (touch.js), an optional lcd (lcd.js) and the flex sensor (flex.js) are all split out to different files.
Installation
Install these scripts and packages on both Edison boards by moving the code on there using the XDK or scp. We found that sometimes the XDK does not compile the dependencies correctly on your development machine, so you might need to copy over the sources and run 'npm install' on the Edison itself. This was specifically the case for different bluetooth libraries we tried.
Handling DDP
An important part of the buddy is that it can receive exercises the kid needs to perform. For this we defined a "collection", a place for documents to be published. Using DDP we subscribe on new messages in this inbox collection:
// At the server: Meteor.publish("userInbox", function() { return Inbox.find({ userId: this.userId }); })
// So the inbox below contains only messages for the logged in user.
ddp.subscribe('userInbox', []);
var taskObserver = ddp.observe("inbox");
taskObserver.added = function(id) {
var message = ddp.collections.inbox[id];
console.log("Do something with", message);
}
Step 4: Install the Server
The server is also made in Node JS, using Meteor. Run this server on the same network as the clients.
https://github.com/hermanbanken/thomas-bot
Once the server is up, the clients can be ran, and you should hear: "Are you ready to play?"
How the server works
The server uses Meteor, which automatically publishes all changes to the database to all connected clients via a special protocol, DDP. Therefore the Edisons always have the latest state. The Edisons can use the server to get to know other buddies to discover using Bluetooth or send messages to other clients.
Meteor also publishes a web interface which is also always up to date. It shows an overview of the buddies with latest online time. The code for the client is located in server/client since it is the client part of the cloud server.
Speech
Currently the server acts as the mouth for the Buddy: it speaks the texts using Apple's speech synthesis software available under the command 'say'. Try for example 'say -v "Alex" "How are you?"' in Terminal on a Mac and hear a voice say this text out loud.
We looked into speech synthesis and we are sure that it is possible to synthesise voice on the Edison itself, but due to time constraints we skipped this. If you feel adventurous you could follow up with this other instructable:
https://www.instructables.com/id/Speaking-Assistant...