Introduction: The Pee Timer: Connecting the Arduino, the Intel Perceptual Computing Camera, and a Submersible Water Pump

By www.facebook.com/austin.toombs
In this Instructables we will walk through how we connected an Intel Perceptual Computing Camera, an Arduino Uno, and a submersible water pump so that the water pump transfers water from one vessel to another when nobody is sitting in front of the camera.

We created this project as an example of Critical Making: making activities that invoke reflection in their observers. It is a design that is intended to highlight and encourage debate surrounding the experiences of blame, shame, guilt, and anxiety that can accompany the use of personal data tracking devices. By building these ideas into a design, we hope to aid people in recognizing and expressing their feelings about these ideas.

Here's how it works: This project takes note of when a computer user walks away from her computer and assumes that the time she is away from the computer is spent using the restroom. If she is gone for too long, it publicly shames her by announcing on Twitter how long she has been away from her computer. At the same time, it records how long she has been gone by pumping water into a small receptacle on her desk, as if the computer itself is urinating. When she returns to her desk, she must dump that water back into the originating receptacle, or risk an overflow that could ruin her desk and the things on it.

The quantified self movement (www.quantifiedself.com) focuses on people collecting, managing, and analyzing data that they collect through technology. When doing this, it is important to keep in mind the side effects of these devices that are not being recorded, but still have an impact on one's physical, emotional, and psychological wellbeing. For example: when tracking hours or minutes of work through a time-tracking application, it can become easier to experience guilt while not working. This may seem like a good thing, because one should be working, and tracking work can benefit efficiency. However, this can have detrimental side effects such as anxiety or feelings of guilt. With the project we describe here, we aim to illuminate those side effects as a means of starting conversations about these complex issues.

One potential source of unnecessary guilt can come from taking necessary breaks, such as using the restroom. Should you track the time you are peeing as work time or break time? Should it only be limited to a standard minimum to maximize efficiency? On the one hand you are not working, but on the other you are completing a task that is necessary for work to continue. This concept is intentionally hyperbolic, with the hope that anyone actually using it will eventually realize "No, this is ridiculous, I should be allowed to pee without feeling bad about it" and then extend that idea to other areas of their lives. Such as "I should be allowed to take time off from work to spend time with my children without feeling guilty," "I should be allowed to read a trashy novel during my break without feeling like I should be working," and, eventually, "I should not feel guilty for making reasonable decisions." By playing with the ideas of work and the quantified self in a playful way we hope to open up a conversation about the nature of work, breaks, and the quantified self.

The rest of this document will be about the process involved in connecting the various technologies we used for this concept. The basic components of which include:
  • Twitter integration with Processing
  • Connecting a Processing application to an Arduino through a serial connection
  • Connecting an Arduino to a water pump through a PowerSwitch Tail

Step 1: Materials and Tools

The materials we used for this project are:

Step 2: Programming

In this step we will cover the programming aspects required for this project. The program is split into two parts:
  • Processing
    • The processing side of the program runs in the 32 bit Processing environment for Windows.
    • It connects to the Intel Perceptual Computing Camera and tracks when the user is sitting in front of the laptop or when the user has left through the face detection built into the camera.
    • It communicates these changes to the Arduino through a serial connection.
    • It also posts "randomized" messages to Twitter, using the Twitter API. These randomized messages are pulled from collections of messages that target specific situations or events, triggered throughout the code. In our particular project, we're using the example of a program that tracks how many times a user gets up to use the restroom, and then chastises the user if they are gone too long.
      • To work through connecting Processing to Twitter, we used this guide: http://blog.blprnt.com/blog/blprnt/updated-quick-t...
      • You will likely find it useful, as it walks you through the process of setting up the application on Twitter, generating authentication keys, etc.
  • Arduino
    • The Arduino side of the program takes the information of when the user leaves and returns to their laptop, and uses it to decide when to turn on the water pump.
    • It also manages the water pump: the particular pump we have actually pumps water too quickly for what we want it to do, so the Arduino program strategically turns the water pump on and off so that we can more easily control the flow of the water without needing to worry about modifying the submersible pump itself.
      • Since we have the water pump plugged into the PowerSwitch Tail, we turn the water pump on and off by turning the PowerSwitch Tail on and off.

The code for both of these applications is provided below. I have included comments throughout the code to help you figure out what is happening for each section, and where you will need to make your own changes. In my opinion, the code is quite difficult to read in the Instructables format, especially since it likes to remove the tabs. If you copy and paste it to Processing or Arduino, it will automatically color-code where appropriate, and make it easier to navigate.

Processing
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
/**
// PeeTracker32
// Developed by Austin Toombs (email: altoombs indiana edu)
// Concept by Austin Toombs and Shad Gross
// January 2014
**/

/** import libraries etc **/
import intel.pcsdk.*;
import processing.serial.*;

/** Initialize variables **/
/** Intel Perceptual Computing Variables **/
PXCUPipeline session;
PImage rgbTex;

int[] faceLabels = {PXCMFaceAnalysis.Landmark.LABEL_LEFT_EYE_OUTER_CORNER,
PXCMFaceAnalysis.Landmark.LABEL_LEFT_EYE_INNER_CORNER,
PXCMFaceAnalysis.Landmark.LABEL_RIGHT_EYE_OUTER_CORNER,
PXCMFaceAnalysis.Landmark.LABEL_RIGHT_EYE_INNER_CORNER,
PXCMFaceAnalysis.Landmark.LABEL_MOUTH_LEFT_CORNER,
PXCMFaceAnalysis.Landmark.LABEL_MOUTH_RIGHT_CORNER};

ArrayList facePts = new ArrayList();
ArrayList faceBoxes = new ArrayList();

/** Collections of potential tweets for verious situations
// If you create your own collections of Strings to pull from, this is where you can initialize them
**/
ArrayList returnTweets = new ArrayList();
ArrayList firstThresholdTweets = new ArrayList();
ArrayList secondThresholdTweets = new ArrayList();
ArrayList thirdThresholdTweets = new ArrayList();

/** booleans for state logic within the program **/
boolean faceState = false; //whether or not a face is seen
boolean prevFaceState = false; //whether or not a face was seen in the previous

/** variables for integrating with Twitter **/
Twitter twitter;

/** variables for monitoring how long the user has been gone
// we set up various "milestones" for while the user s gone. We have 3 here, but you can
// changes these easily by adding more here and throughout the code where these milestones
// are referenced
**/
long savedTime;
int minimumTime = 6000; //minimum time limit before sending a message
int firstMilestone = 30000; //first milestone for sending shameing messages
boolean firstMilestoneMessageSent = false; //keeping track of if we sent the first milestone message
int secondMilestone = 60000; //second milestone for sending shameing messages
boolean secondMilestoneMessageSent = false; //keeping track of if we sent the second milestone message
int thirdMilestone = 90000; //third milestone for sending shaming messages
boolean thirdMilestoneMessageSent = false; //keeping track of if we sent the third milestone message

/** variables for serial connection **/
Serial myPort; //the Serial port object


void setup()
{
//set the size of the canvas
size(640,480);

//create image from camera information (not necessary)
rgbTex = createImage(640,480,RGB);

//the session information for the Intel Perceptual Computing Camera
session = new PXCUPipeline(this);
session.Init(PXCUPipeline.COLOR_VGA|PXCUPipeline.FACE_LOCATION|PXCUPipeline.FACE_LANDMARK);

//initializing the connection to Twitter *complete with authentication!
// You will need to generate your own Twitter application keys and include them here
ConfigurationBuilder cb = new ConfigurationBuilder();
cb.setOAuthConsumerKey("XXXXXXXXXXXXXXXXXXXX");
cb.setOAuthConsumerSecret("YYYYYYYYYYYYYYYYYYYYYY");
cb.setOAuthAccessToken("ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ");
cb.setOAuthAccessTokenSecret("XYZYXYZXYZYXYZXZXYZYXYZYXYZYXYZ");

//initialize twitter object
twitter = new TwitterFactory(cb.build()).getInstance();

//this doesn't have to be called at all, but is fun to see
//commented out for now: getPreviousTweets();

//initialize serial port and set baud rate to 9600
myPort = new Serial(this, Serial.list()[0], 9600);

//initialize the collections of tweets
initializeTweets();
}

//default draw function in every processing program
void draw()
{
/////// Getting data from the camera and drawing to the canvas ////////
// before playing with perceptual computing camera, we have to make sure that
// it can actually see things. Everything in this if statement should
// remain as-is for now. We use it to determine if a face is even within
// view. There is likely an easier way to do this, but we like to see the face
// boxes so we can tell what it sees as a face and what it does not
if(session.AcquireFrame(false))
{
session.QueryRGB(rgbTex);
facePts.clear();
faceBoxes.clear();

for(int i=0;;++i)
{
long[] ft = new long[2];
if(!session.QueryFaceID(i,ft))
break;

PXCMFaceAnalysis.Detection.Data fdata = new PXCMFaceAnalysis.Detection.Data();
if(session.QueryFaceLocationData((int)ft[0], fdata))
{
faceBoxes.add(fdata.rectangle);

PXCMFaceAnalysis.Landmark.LandmarkData lmark = new PXCMFaceAnalysis.Landmark.LandmarkData();
for(int f=0;f
{
if(session.QueryFaceLandmarkData((int)ft[0],faceLabels[f], 0, lmark))
{
facePts.add(lmark.position);
}
}
}
}
session.ReleaseFrame();
} //end of face detection if statement

//displays what the camera sees on the canvas
image(rgbTex,0,0);
pushStyle();
stroke(255);
noFill();

//draws the face boxes on the screen
for(int f=0;f
{
PXCMRectU32 faceLoc = (PXCMRectU32)faceBoxes.get(f);
rect(faceLoc.x,faceLoc.y,faceLoc.w,faceLoc.h);
}
fill(0,255,0);
for(int g=0;g
{
PXCMPoint3DF32 facePt = (PXCMPoint3DF32)facePts.get(g);
ellipse(facePt.x,facePt.y,5,5);
}
popStyle();

////// That's the end of the drawing stuff ////////

//save the previous face state (for tracking purposes)
prevFaceState = faceState;

//if we can't even see a face at all
if(faceBoxes.size()==0){
faceState = false; //we can't see the face, so the user isn't there

//if they were there a moment ago, and they just now left
if(prevFaceState == true){
//output that camera no longer sees a face
System.out.println("Can't see a face, starting timer");

//start tracking time. millis() measure the time in milliseconds
// since the start of the program
savedTime = millis();

//send signal to serial that we cannot see a face
myPort.write('2');
}

//the time the user has been gone is the current time in milliseconds
// since the start of the program minus the time in milliseconds since
// the start of the program when the user left.
long timeAway = millis() - savedTime;

//use the timeAway information to determine when to send message
processAwayAlerts(timeAway);
}
else {
//if we see faces, we aren't timing them
faceState = true;

//if we see a face where we previously didn't, then we know
// they just got back
if(prevFaceState == false){
//calculate how long the user has been gone total
long timeAway = millis() - savedTime;

processJustReturnedAlerts(timeAway);


resetMessageBooleans();
}
}
//////// end of the draw method /////////
}

//set up each of the collections of tweets that can be used for
// specific situations
void initializeTweets(){
returnTweets.add("@altoombs was away from his desk for XXX seconds");
returnTweets.add("@altoombs peed for XXX seconds");
returnTweets.add("@altoombs was unproductive for XXX seconds");
returnTweets.add("@altoombs managed to procrastinate for XXX seconds");
returnTweets.add("@altoombs slacked off for XXX seconds");
returnTweets.add("@altoombs was completely useless for XXX seconds");

firstThresholdTweets.add("@altoombs is probably peeing right now, so if you need something check back soon");
firstThresholdTweets.add("@altoombs has been away from his desk for some time now. Don’t make a habit of this");
firstThresholdTweets.add("@altoombs will be back soon. Or he better be.");
firstThresholdTweets.add("@altoombs can’t come to the phone right now. Unless he has it with him. In the bathroom.");

secondThresholdTweets.add("@altoombs … where are you? Why have you been away for so long?");
secondThresholdTweets.add("@altoombs whatever you’re doing right now better be freaking important.");
secondThresholdTweets.add("@altoombs if you aren’t going to work today you might as well just go home");
secondThresholdTweets.add("@altoombs might have a medical problem. It should not take this long to pee.");

thirdThresholdTweets.add("@altoombs takes forever to pee. He better finish up quick or risk wasting his life away");
thirdThresholdTweets.add("@altoombs is wasting his time. Everyone should pressure him to get back to his desk");
thirdThresholdTweets.add("@altoombs we’re sending out the search party. You have been gone too long.");
thirdThresholdTweets.add("@altoombs is not a model for how you should behave at work.");
}

//determine which messages to send and when while the user is gone
// based on the time they have been away
void processAwayAlerts(long timeAway){
//if the time they have been gone is longer than the minimum
// time we are concerned about, then turn on the LED
if(timeAway > minimumTime){
myPort.write('1');
}

//if the time they have been gone is longer than the first
// milestone time, then take appropriate measures
if(timeAway > firstMilestone && !firstMilestoneMessageSent){
//send the message (either over twitter or to console)
sendMessage(randomTweet(firstThresholdTweets));

//mark that the first milestone message has been sent
firstMilestoneMessageSent = true;
}

//if the time they have been gone is longer than the second
// milestone time, then take appropriate measures
if(timeAway > secondMilestone && !secondMilestoneMessageSent){
//send the message (either over twitter or to console)
sendMessage(randomTweet(secondThresholdTweets));

//mark that the second milestone message has been sent
secondMilestoneMessageSent = true;
}

//if the time they have been gone is longer than the third
// milestone time, then take appropriate measures
if(timeAway > thirdMilestone && !thirdMilestoneMessageSent){
//send the message (either over twitter or to console)
sendMessage(randomTweet(thirdThresholdTweets));

//mark that the second milestone message has been sent
thirdMilestoneMessageSent = true;
}
}

//determine what to do when the user returns
void processJustReturnedAlerts(long timeAway){
//only display message if they were gone the minimum amount of
// time. Otherwise they were probably just looking away or scratching
// their face or fixing their hair or... you get it
if(timeAway > minimumTime){
//report to console how long the user was gone
System.out.println("User just returned after: " + timeAway + " miliseconds.");

//send message either over twitter or over console
sendMessage(randomTweet(returnTweets).replace("XXX", ""+timeAway/1000));
}

//send a signal over serial to tell the Arduino that the user
// is in front of the computer
myPort.write('0');
}

//choose a random tweet from the given tweet collection and
// return that message as a String
String randomTweet(ArrayList tweetList){
//choose a random number based on the length of the collection
int tweetIndex = (int) random(tweetList.size());

//return the String at that location
return tweetList.get(tweetIndex);
}

//send a message either over twitter or to console
void sendMessage(String message){
//send the message over twitter (comment out while testing)
/**try{
twitter.updateStatus(message);
}
catch (TwitterException te) {
println("Couldn't connect: " + te);
};**/

//send the message to console
System.out.println(message);
}

//reset the message booleans. Otherwise the threshold messages could only
// be sent once during each run of the program
void resetMessageBooleans() {
firstMilestoneMessageSent = false;
secondMilestoneMessageSent = false;
thirdMilestoneMessageSent = false;
}

/**
* This method just finds the previous tweets made by this device and
* prints them out to the console. It can be removed and simply not
* called and won't change any other functionality of this program
**/
void getPreviousTweets() {
Query query = new Query("#peeTimer");
//query.setRpp(100); //doesn't work w/most recent Twitter4j

//Try making the query request.
try {
//the following line updates the twitter status
QueryResult result = twitter.search(query);
ArrayList tweets = (ArrayList) result.getTweets();

for (int i = 0; i < tweets.size(); i++) {
Status t = (Status) tweets.get(i);
User u = (User) t.getUser();
String user = u.getName();
String msg = t.getText();
println("Tweet by " + user + ": " + msg);
};
}
catch (TwitterException te) {
println("Couldn't connect: " + te);
};
}
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------


Arduino
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
/**
// peetimerArduinoSketch
// Developed by Austin Toombs (email: altoombs indiana edu)
// Concept by Austin Toombs and Shad Gross
// January 2014
**/

//variables for reading to and from serial
char val; // Data received from the serial port

//pin assignments
int ledPin = 13; // Set the pin to digital I/O 13
int waterPumpPin = 10; // This pin actually controls the PowerSwitch Tail, turning it on and off

//time tracking for water pump
long previousMillis = 0;
long offInterval = 4000; //how long we need it off per interval
long onInterval = 2000; //how long we need it on per interval
boolean timeStarted = false;

//tracking the state of the water pump
int waterPumpState = LOW;

void setup()
{
pinMode(ledPin, OUTPUT); // Set pin as OUTPUT
pinMode(waterPumpPin, OUTPUT); // set pin as OUTPUT
//initialize serial communications at a 9600 baud rate
Serial.begin(9600);
}

void loop() {
if (Serial.available())
{ // If data is available to read,
val = Serial.read(); // read it and store it in val
}
if (val == '1')
{ // If 1 was received
digitalWrite(ledPin, HIGH); // turn the LED on

// if the timer has not started yet, start it
if(!timeStarted) { timeStarted = true; }

// decide what to do with the water pump, based on time
manageWaterPump();
} else { // if anything other than a 1 is received
digitalWrite(ledPin, LOW); // turn off the LED
digitalWrite(waterPumpPin, LOW); //turn off the water pump
timeStarted = false; // turn off the timer
}
delay(10); // Wait 10 milliseconds for next reading
}

//decide how to manage the water pump
void manageWaterPump(){
//the current amount of time that has passed since we received the 1 in serial
unsigned long currentMillis = millis();

//if the water pump is already off
if(waterPumpState == LOW){
//and it has been off long enough, then turn it on
if(currentMillis - previousMillis > offInterval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
waterPumpState = HIGH;

// set the LED with the ledState of the variable:
digitalWrite(waterPumpPin, waterPumpState);
}
}
else { //if the water pump is already on
// and it has been on long enough, then turn it off
if(currentMillis - previousMillis > onInterval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
waterPumpState = LOW;

// set the LED with the ledState of the variable:
digitalWrite(waterPumpPin, waterPumpState);
}
}
}
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------


Step 3: Setup and Testing

Intel Perceptual Computing Camera
Setting up the Intel Perceptual Computing Camera (or the Intel Real Sense 3D Camera) is fairly straightforward and the process is described pretty well here: http://software.intel.com/en-us/vcsource/tools/per...

When you download the SDK, it comes with a lot of great examples that you can pull from to help you get started. Our program uses some simple face detection, which we stripped from the face tracking example "FaceTracking.pde"

PowerSwitch Tail II
The PowerSwitch tail takes a signal from the Arduino when it needs to be switched on and off. So we have to connect it to the Arduino. For our project we did this by first opening the ports for "in +" and "in -" with a 3.0 mm flat head screwdriver. (Do NOT use the "ground" port.) Then we stuck a jumper wire into each port and tightened them to hold the end of the jumper wire in place. These jumper wires then need to be plugged into the Arduino. According to the Arduino code, we use pin 10 to control the PowerSwitch Tail II, so the wire plugged into the "in +" port of the PowerSwitch Tail II needs to plug into pin 10, and the wire from the "in -" port should connect to one of the Arduino's ground pins.

Water Pump
We are controlling the water pump through the PowerSwitch Tail II by giving and taking away power when the Arduino signals it, so setting up the water pump is remarkably easy. First, set the water pump to the lowest setting if you are using the same pump as we did, otherwise it pumps water too quickly. Then, plug it into the PowerSwitch Tail II and attach the plastic tube, making sure the tube can reach the receiving receptacle. Finally, place the water pump into the holding receptacle with the water.

Arduino
Connecting the Arduino to Windows through Parallels on a Mac can be a little bit tricky, but is mostly very similar to connecting it to a PC. If you are also using Windows on a Mac through Parallels, make sure that you connect the Arduino AFTER you start up Parallels and Windows, so that you can choose to connect it to the Windows virtual machine, allowing it to communicate with it so that Processing can see it as a potential serial communication partner.

Step 4: Finished!

After all of the code is uploaded to the Arduino, all you have to do is run the Processing sketch and watch as the Intel Perceptual Computing Camera tracks your face and determines when you get up from your computer. After being away from the computer for around 6 seconds (a buffer we built into the program so that it wouldn't activate every time you scratched your forehead), the water pump will kick on. The current program is set so that the water pump runs for 2 seconds and pauses for 4 seconds. This is to keep the pump from pumping too quickly, but these variables can be changed around in the Arduino code.

We would like to acknowledge the following people for their funding, support, help or inspiration for this project: Intel ISCT Social Computing, NSF Creative IT grant, Shaowen Bardzell, Jeffrey Bardzell, Indiana University School of Informatics and Computing, Critical Making Hackathon: Situated Hacking, Surveillance and Big Data workshop organizers (Karen Tanenbaum, Josh Tanenbaum, Amanda Williams, Matt Ratto, Gabriel Resch, and Antonio Gamba Bar), The Intel Perceptual Computing Group, Derek Whitley, and Bloominglabs.

We hope you found this useful! Thanks for reading along.

Austin Toombs and Shad Gross