Introduction: How to Do Cat Flap Monitoring

By
About: My goal is to showcase how you can use simple AI with some tools to make projects that can be challenging if done by hand (sound classification, gestures recognition, event detection ...)

When in vacation, it could be a good addition to be able to know if my cats are at home or not. To do that we will to add detection to the cat flap, to know if a cat is entering or exiting.


We will monitorate the vibration made by the cat slap with an accelerometer and do classification with simple artificial intelligence (AI). The advantage of using AI in this case is that it is easier. I know, it is crazy, but in this case, with the tool that we will be using, it is easier and faster to use AI than doing it manually by setting threshold and so on.


To release this use case, we will:

  1. Write some Arduino code to use the accelerometer
  2. Log vibration data of the cat entering and exiting using serial
  3. Use NanoEdge AI Studio tool to get an AI library (a model + code to use it)
  4. Add the detection in few lin to the Arduino code

That it!

Supplies

Hardware:

  • Arduino Uno R4 WIFI
  • X-NUCLEO-IKS01A3 with LSM6DS0 3 axes accelerometer.
  • A micro-USB cable to connect the Arduino board to your desktop machine


Software:

  • Arduino IDE
  • NanoEdge AI Studio: it is a free software developed by STMicroelectronics to easily create embedded AI libraries (preprocessing + model) based on data that you import.

Step 1: Setup

First plug the shield on the board:


In Arduino IDE:

Make sure you selected the right COM port: Tools > Port and select the right one.

Select the right board:

  • Tools > Boards > Arduino Renesas UNO R4 boards > Arduino UNO R4 WIFI
  • If you don't find it, click on Tools > Boards > Boards Manager..., look for the UNO R4 and install the package

Select the needed libraries:

  • Sketch > Include Library, we need to include the following libraries:
  • Wire
  • LIS2DW12Sensor
  • You can also download libraries that you don't have: Sketch > Include Library > Manage libraries...
  • Later in this tutorial (Step 8) we will also need to add the NanoEdge AI Library to process the sound on its own instead of doing it manually.


Cat slap setup:

  • Plug the X-NUCLEO-IKS01A3 on the Arduino R4 WiFI
  • Put the setup on the cat slap to later collect vibrations

Step 2: Using the Accelerometer

Our goal is to collect data to train an AI to know if a cat is entering or exiting the house.

To achieve that, we need to use the accelerometer.

We need to:

  • Include the needed libraries
  • Define the sensor's parameters
  • Initialize the Serial and I2C and accelerometer
  • Create a buffer and collect data

You will see in the following code that we collect buffers of data in a variable neai_buffer (for NanoEdge AI buffer) not simply samples of 3 values (x,y,z). It is to later create the AI model.

Why buffers ? Because they contains the information of what happened during a period of time. With buffers (signals) you can answer question like "is it increasing or decreasing", but with just a value you cannot.

We will use buffer that contains example of vibrations made by a cat entering a house and other buffer containing example of vibrations made by the cat exiting the house and classify them automatically with the AI model.


You can flash this code to collect buffer of size 1024 * 3 axes and print them through Serial:

/* Libraries part */
#include "Wire.h"
#include <LIS2DW12Sensor.h>


/* Macros definitions */
#define SERIAL_BAUD_RATE  115200


/* Sensor data rates.
   You can choose from the following values for both accel & gyro:
   12.5f, 25.0f, 50.0f, 100.0f, 200.0f, 400.0f, 800.0f & 1600.0f.
*/
#define SENSOR_DATA_RATE	1600.0f


/* Sensor ranges.
   You can choose from:
   2, 4, 8 & 16.
*/
#define SENSOR_RANGE	4

#define SENSOR_SAMPLES	1024
#define AXIS  3


/* Sensor object declaration using I2C */
LIS2DW12Sensor Accelero(&Wire);


/* Global variables definitions */
static uint8_t drdy = 0;
static uint16_t neai_ptr = 0;
static int32_t accelerometer[3];
static float neai_buffer[SENSOR_SAMPLES * AXIS] = {0.0};


/* Initialization function: In this function,
    code runs only once at boot / reset.
*/
void setup() {
  /* Init serial at baud rate 115200 */
  Serial.begin(SERIAL_BAUD_RATE);


  /* I2C workaround: Sometimes, on some boards,
     I2C get stuck after software reboot, reset so,
     to avoid this, we toggle I2C clock pin at boot.
  */
  pinMode(SCL, OUTPUT);
  for (uint8_t i = 0; i < 20; i++) {
    digitalWrite(SCL, !digitalRead(SCL));
    delay(1);
  }
  delay(100);


  Wire.begin();
  Accelero.begin();
  Accelero.Enable_X();
  Accelero.Set_X_ODR(SENSOR_DATA_RATE);
  Accelero.Set_X_FS(SENSOR_RANGE);
}


/* Main function: Code run indefinitely */
void loop() {
  // Those values should be adapted regarding the user setup
  // Goal is to complete the buffer only if a cat is there 
  if (!(rms < 700 || rms > 2000))
    return;
  /* Get data in the neai buffer */
  while (neai_ptr < SENSOR_SAMPLES) {
    /* Check if new data if available */
    Accelero.ReadReg(LIS2DW12_STATUS, &drdy);
    if (drdy & 0x01) {
      /* If new data is available we read it ! */
      Accelero.Get_X_Axes(accelerometer);
      /* Fill neai buffer with new accel data */
      neai_buffer[AXIS * neai_ptr] = (float) accelerometer[0];
      neai_buffer[(AXIS * neai_ptr) + 1] = (float) accelerometer[1];
      neai_buffer[(AXIS * neai_ptr) + 2] = (float) accelerometer[2];
      /* Increment neai pointer */
      neai_ptr++;
    }
  }
  /* Reset pointer */
  neai_ptr = 0;


  /* Print the whole buffer to the serial */
  for (uint16_t i = 0; i < AXIS * SENSOR_SAMPLES; i++) {
    Serial.print((String)neai_buffer[i] + " ");
  }
  Serial.print("\n");
  // }


  /* Clean neai buffer */
  memset(neai_buffer, 0.0, AXIS * SENSOR_SAMPLES * sizeof(float));
}

Step 3: Nanoedge AI Studio

With the code in the previous step, we collect buffers of size 1024 with a data rate of 1.6kHz. Which mean that when we have a vibration above the small threshold, we collect the data representing 64ms of the cat going through the cat slap.


We will collect the dataset and create the model directly in NanoEdge (it is free):

  • Install NanoEdge AI Studio: https://stm32ai.st.com/download-nanoedgeai/
  • Create N class classification project
  • Select accelerometer 3 axis as a sensor
  • Select Arduino UNO R4 WIFI as target
  • Collect data:
  • Flash code from the previous step with Arduino IDE
  • In NanoEdge, in signals tab, click add Signal, then serial
  • First collect multiple examples of a cat entering the house (100 should be enough)
  • Then multiple examples of the cat exiting

It can be quite tedious to make the cat do what you want, you may try to do it with your hand for a first prototype, but it will work better if done with a real cat :)

  • Launch a benchmark and obtain a good accuracy

A entire benchmark can take hours, but in the first few minutes, you will have a score close to the final one. So if in few minutes you have 90+% of score, you can stop it if you want.

If you have less than 80+%, you may need to collect more / better data

  • Compile the library

In the compilation step, you will get a .zip containing a library for Arduino. More on that in the next step.


Step-by-step tutorial for more details on how to use NanoEdge AI Studio and Arduino: https://wiki.st.com/stm32mcu/wiki/AI:How_to_create_Arduino_Rock-Paper-Scissors_game_using_NanoEdge_AI_Studio

Step 4: Add Classification to the Code

Now that we have the classification library, we need to add it to our Arduino code:

  • Open the .zip obtained, there is an Arduino folder containing another zip
  • Import the library in Arduino IDE: Sketch > Include library > Add .ZIP library... and select the .zip in the Arduino folder


Here is a sample example of code to demonstrate what we need to do (the code doesn't compile, it is just for explanation purpose)

The full code is in the next step.

#include "NanoEdgeAI.h"
#include "knowledge.h"

#define CLASS_NUMBER 2

/* NanoEdgeAI variables part */
uint8_t neai_code = 0; //to check the initialization of the library
uint16_t id_class = 0; // Point to id class (see argument of neai_classification fct)
float output_class_buffer[CLASS_NUMBER]; // Buffer of class probabilities
const char *id2class[CLASS_NUMBER + 1] = { // Buffer for mapping class id to class name
  "unknown", //always present
  "inside",
  "outside",
};
static float neai_buffer[SENSOR_SAMPLES * AXIS] = {0.0}; //our neai buffer

void setup() {
  ...
  /* initialize NanoEdge Library */
  neai_code = neai_classification_init(knowledge);
  if (neai_code != NEAI_OK) {
    Serial.print("Not supported board.\n");
  }
  ...
}

void loop() {  
   ...
   /* make a classification */
   get_microphone_data();
   neai_classification(neai_buffer, output_class_buffer, &id_class);
   /* then depending on the class in id_class, we play up, down or wathever */
   ...
}

Warning:

You need to check that the id2class variable in the Arduino code is the same than in NanoEdgeAI.h file from the library we imported earlier.

Step 5: Complete Code

You can find the complete code attached or below:

The code contains everything explained above


/* Libraries part */
#include "Wire.h"
#include <LSM6DSOSensor.h>
#include <NanoEdgeAI.h>
#include <knowledge.h>


/* Macros definitions */
#define SERIAL_BAUD_RATE  115200


/* Define the data type you want to collect */
#define ACCELEROMETER 1
#define GYROSCOPE 0
#define SENSOR_DATA_TYPE	ACCELEROMETER


/* Sensor data rates.
   You can choose from the following values for both accel & gyro:
   12.5f, 26.0f, 52.0f, 104.0f, 208.0f, 417.0f, 833.0f & 1667.0f.
*/
#define SENSOR_DATA_RATE	417.0f


/* Sensor ranges.
   You can choose from:
   2, 4, 8 & 16 for accelerometer.
   125, 250, 500, 1000 & 2000 for gyroscope.
*/
#define SENSOR_RANGE	2


/* NanoEdgeAI defines part
   NEAI_MODE = 1: NanoEdgeAI functions = AI Mode.
   NEAI_MODE = 0: Datalogging mode.
*/
#define NEAI_MODE 1
#define SENSOR_SAMPLES	1024
#define AXIS  3
#define CLASS_NUMBER 2


/* Sensor object declaration using I2C */
LSM6DSOSensor AccGyr(&Wire);


/* Global variables definitions */
static uint8_t drdy = 0;
static uint16_t neai_ptr = 0;
static int32_t sensor_values[3];
static float neai_buffer[SENSOR_SAMPLES * AXIS] = {0.0};


/* NEAI library variables */
// Buffer of class probabilities
uint16_t id_class = 0; // Point to id class (see argument of neai_classification fct)
float output_class_buffer[CLASS_NUMBER]; // Buffer of class probabilities
const char *id2class[CLASS_NUMBER + 1] = { // Buffer for mapping class id to class name
  "unknown",
  "inside",
  "outside",
};
static uint8_t neai_code = 0;


/* Initialization function: In this function,
    code runs only once at boot / reset.
*/
void setup() {
  /* Init serial at baud rate 115200 */
  Serial.begin(SERIAL_BAUD_RATE);


  /* I2C workaround: Sometimes, on some boards,
     I2C get stuck after software reboot, reset so,
     to avoid this, we toggle I2C clock pin at boot.
  */
  pinMode(SCL, OUTPUT);
  for (uint8_t i = 0; i < 20; i++) {
    digitalWrite(SCL, !digitalRead(SCL));
    delay(1);
  }
  delay(100);


  Wire.begin();
  AccGyr.begin();
  if (SENSOR_DATA_TYPE == ACCELEROMETER) {
    AccGyr.Enable_X();
    AccGyr.Disable_G();
    AccGyr.Set_X_ODR(SENSOR_DATA_RATE);
    AccGyr.Set_X_FS(SENSOR_RANGE);
  }
  else {
    AccGyr.Enable_G();
    AccGyr.Disable_X();
    AccGyr.Set_G_ODR(SENSOR_DATA_RATE);
    AccGyr.Set_G_FS(SENSOR_RANGE);
  }


  /* Initialize NanoEdgeAI AI */
  neai_code = neai_classification_init(knowledge);
  if (neai_code != NEAI_OK) {
    Serial.print("Not supported board.\n");
  }
}


/* Main function: Code run indefinitely */
void loop() {


  // Here we get the values of accelerations and calcul the RMS
  AccGyr.Get_X_Axes(sensor_values);
  int rms = 0 ;
  rms = sqrt(sensor_values[0] * sensor_values[0] + sensor_values[1] * sensor_values[1] + sensor_values[2] * sensor_values[2]);


  // Those values should be adapted regarding the user setup
  // Goal is to complete the buffer only if a cat is there
  if (!(rms < 700 || rms > 2000))
    return;
  /* Get data in the neai buffer */
  while (neai_ptr < SENSOR_SAMPLES) {
    /* Check if new data if available */
    if (SENSOR_DATA_TYPE == ACCELEROMETER) {
      AccGyr.Get_X_DRDY_Status(&drdy);
    }
    else {
      AccGyr.Get_G_DRDY_Status(&drdy);
    }
    if (drdy) {
      /* If new data is available we read it ! */
      if (SENSOR_DATA_TYPE == ACCELEROMETER) {
        AccGyr.Get_X_Axes(sensor_values);
      }
      else {
        AccGyr.Get_G_Axes(sensor_values);
      }
      /* Fill neai buffer with new accel data */
      neai_buffer[AXIS * neai_ptr] = (float) sensor_values[0];
      neai_buffer[(AXIS * neai_ptr) + 1] = (float) sensor_values[1];
      neai_buffer[(AXIS * neai_ptr) + 2] = (float) sensor_values[2];
      /* Increment neai pointer */
      neai_ptr++;
    }
  }
  /* Reset pointer */
  neai_ptr = 0;
  /* Depending on NEAI_MODE value, run NanoEdge AI functions
     or print accelerometer data to the serial (datalogging)
  */
  if (NEAI_MODE) {
    neai_classification(neai_buffer, output_class_buffer, &id_class);
    Serial.print((String)"Class detected " + id_class + ".\n");
  } else {
    /* Print the whole buffer to the serial */
    for (uint16_t i = 0; i < AXIS * SENSOR_SAMPLES; i++) {
      Serial.print((String)neai_buffer[i] + " ");
    }
    Serial.print("\n");
  }


  /* Clean neai buffer */
  memset(neai_buffer, 0.0, AXIS * SENSOR_SAMPLES * sizeof(float));
}


Attachments

Step 6: To Go Further

Now that you can classify if a cat is entering or exiting the house, it is up to you to define what to do with it.

Here are some ideas:



Thank you for reading :)