In this tutorial, I'll walk you through designing a glove that lets you interact with music through dance. You'll build an accelerometer-enabled glove, design a composition in Ableton, and then connect the two in as complex or simple a way as you'd like!

Supplies

Ableton (or free trial)
An Arduino
Jumper cables
Soldering iron
Cardboard
Hot glue gun
Lots of imagination

Step 1: Concept

This project is designed to be fun. If the way the example project in this tutorial works isn't fun for you, redesign it!

I recommend putting on some of your favorite songs, moving your hands to them, and seeing what happens. Do you move your hands up-and-down? Side-to-side? Slowly or quickly? What aspects of music make you want to move your hands? If you have a list of these written down, you'll probably be able to figure out some ways to incorporate the motions you enjoy into your eventual algorithms.

Here are the motions I used:

A quick up-and-down motion triggers the start of the song, drums, or bass. (These happen at different points in the song, not necessarily simultaneously!)
A slow, tilty side-to-side motion triggers a more echoey, high-pitched sound.
At one particular section of the song, tilting my hand upwards makes the music quiet down -- so I've "caught" it in my closed fist.

Use these or make your own!

(Please note: this tutorial doesn't cover how to generate music or melodies live in Ableton! If you stick to these instructions, you'll only be able to increase/decrease volume of tracks or the application of audio effects.)

Step 2: Prepare the Accelerometer

First, figure out what type of accelerometer you have. I used this one; any three-axis accelerometer will do. (Or try another type of sensor if you want to go wild.) Make sure you know how to read the accelerometer data from the Arduino. You may need to download a library for your accelerometer if it uses anything more complex than analog input.

After having tested it with a breadboard, solder short color-coded wires into the pins of your accelerometer. Put a red wire in the power pin, a black wire in the ground pin, and any other necessary wires for accelerometer communication. (If you have an I2C accelerometer, this will be the SCL and SDA pins. If you have an analog accelerometer, there will likely be one pin for each of the x, y, and z outputs.) Make sure your solder is solid and that the beads don't overlap between adjacent pins.

Step 3: Build the Glove

Cut a piece of thin cardboard or thick paper into a rectangle slightly larger than your accelerometer. Glue the accelerometer to the cardboard, making sure that you are putting glue on the bottom. Then, glue the cardboard-backed accelerometer to the back of your glove. Sew each wire loosely to the wrist of the glove to relieve tension on the accelerometer, and then your glove is ready. Connect it to longer wires in order to have enough space to move your hand when it is plugged in.

Step 4: Compose in Ableton

Now it's time to compose the song you'll eventually use the glove to control. I recommend Ableton loops that all sound good together, but can be used to build up gradually: try melody, chords, bass, and percussion. You'll be able to use your glove to control when each loop plays or doesn't.

If you can think of any interesting kinds of sounds to occasionally incorporate into a song, like a weird sound effect or unconventional instrument, try adding in one or two of those as well! You can tie them to less-common hand motions to bring in something interesting every once in a while.

Here is a link to my Arduino-compatible composition, in case you don't want to write one of your own: https://drive.google.com/open?id=10Q9c3XqEX3qZGI1d...

(Unfortunately, teaching you Ableton is not within the scope of the tutorial. However, there are a lot of good how-to videos out there, and Ableton has a 90-day free trial! I recommend this video.)

Step 5: Start Using Firmata

In order to enable your Arduino to communicate with Ableton, you'll need to use a library called Firmata. You'll also need to download the Connection Kit for Ableton.

In Ableton, click Packs>Connection Kit>Devices in the menu on the top left, and then double click on the first device (Arduino) to add it. Make sure you remember which Ableton track you added the device to!

Step 6: Test Firmata

First, we'll test and make sure your Arduino is communicating with Ableton. Upload this code snippet to your Arduino and run it:

#include <Firmata.h>

void analogWriteCallback(byte pin, int value)
{
  if (IS_PIN_PWM(pin)) {
    pinMode(PIN_TO_DIGITAL(pin), OUTPUT);
    analogWrite(PIN_TO_PWM(pin), value);
  }
}

void setup() {
  Firmata.setFirmwareVersion(FIRMATA_FIRMWARE_MAJOR_VERSION, FIRMATA_FIRMWARE_MINOR_VERSION);
  Firmata.attach(ANALOG_MESSAGE, analogWriteCallback);
  Firmata.begin(57600);

}

void loop() {
  Firmata.sendAnalog(0, 800);

}

This is the bare minimum needed to communicate with Firmata. It continuously sends an output of 800 (out of 1024) to port 0 of the Firmata device in Ableton. If you upload this code to your Arduino while you have a Firmata device open in Ableton, it should look like the image above. (Map port 0 to anything in Ableton to be able to see the values.)

You can click the Map button and then any Firmata-compatible device in Ableton to add a mapping between the input received to that port and the value of that Ableton device. Easy examples include the volume of any track or any dial within an audio effect. Explore and see what you can find to map to!

Step 7: Influence Music With Your Hand Movements!

By this time, you should have some music in Ableton, a Firmata script on your Arduino, and an accelerometer glove attached. Let's make some music!

Map ports of the Arduino device in Ableton to different things (I suggest track volume), and then add lines of code to send data to each port from the Arduino.

Firmata.sendAnalog(port, volumeLevel);

Use code like this for each Firmata port.

If you want to do something simple, you can send the accelerometer values unprocessed to Ableton ports and map them from there. For a more sophisticated experience, you can decide: what accelerometer values should trigger sounds, how, and when?

Then play all of your Ableton loops, run your Arduino code, and dance away!

(Disclaimer: if you are planning to create any kind of complex algorithm for your song, it may take a lot of time to fine-tune. "Dance away" may be less accurate than anticipated.)

Step 8: The Track Class (bonus!)

If you don't mind volume popping or have another way to mitigate it, skip this step. Otherwise, read on!

I noticed that switching volume from muted to full in one go creates some unpleasant popping sounds, and it's nice to be able to fade in volume more gradually. However, it's hard to do this in the Arduino's synchronous programming environment. So here is some code to make popping go away:

class Track
{
  public:
  int volume;
  int volumeGoal;
  int updateSpeed;

  Track()
  {
    volume = 0;
    volumeGoal = 0;
    updateSpeed = 0;
  }

  void setVolumeGoal(int goal)
  {
    volumeGoal = goal;
  }
  int getVolumeGoal()
  {
    return volumeGoal;
  }
  void setUpdateSpeed(int fastness)
  {
    updateSpeed = fastness;
  }
  int getVolume()
  {
    return volume;
  }


  void updateVolume()
  {
    if((volume > volumeGoal) && ((volume - volumeGoal) >= updateSpeed))
    {
      volume -= updateSpeed;
    }
    else if((volume < volumeGoal) && ((volumeGoal - volume) >= updateSpeed))
    {
      volume += updateSpeed;
    }
  }
  void mute(int fastness)
  {
    volumeGoal = 50;
    updateSpeed = fastness;
  }
  void full(int fastness)
  {
    volumeGoal = 950;
    updateSpeed = fastness;
  }
};

Each Track has a current volume, a goal volume, and a speed at which it is moving towards that goal volume. When you want to change the volume of a track, call setVolumeGoal(). Every time you run the loop() function in your Arduino, call updateVolume() on every track, and then send that information to Firmata with getVolume(). Change the update speed for faster or more gradual fadeouts! Also, avoid setting the volume to 0 if you can; instead, set it to a very low value (the default in mute() is 100).

Step 9: Track Length, Beats, and More (bonus!)

You can do a lot of things to make the sound resulting from your project be easier to listen to. Here are a few options:

You can track how long the song has been running. In order to do this, you'll have to figure out when the song started; I recommend a while loop in the setup() function which delays your code from running until it has sensed a hand motion. Store the song's start time in a variable using millis(), and check how long it has been going on each time you loop(). You can use this to enable or disable certain features at certain times of the song.

If you know how long your loops are in milliseconds, you can also track how many loops you have been through for a more nuanced understanding of song structure!

Another potential issue you may encounter is when to start and stop a track from playing. I solved this by keeping track of which beat of a measure the song was currently in. Then I could play tracks for any number of beats after a gesture, instead of cutting it off immediately. This makes things flow a lot more smoothly. Here's an example:

if(millis() - lastLoop >= 4000)
  {
    loops += 1;
    lastLoop = millis();
    for(int j = 0; j < 8; j++)
    {
      beatNow[j] = false;
    }
  }
  beat = (millis() - lastLoop) / 250;
  if(beat != lastBeat)
  {
    lastBeat = beat;
    beatsLeft -= 1;
  }

Make sure you update volumes accordingly to the values of beatNow[beat] and/or beatsLeft. Example code incorporating nearly everything in this tutorial, plus some, is attached in case you want to see it in practice.

demo.ino
Download

Introduction: Dance Glove