Introduction: VisionFit: ML and Computer Vision Based Personal Trainer Which Detect the Posture of the User and Then Give Laser Guidance for Body Positioning

Vision Fit addresses the growing need for accessible, reliable exercise guidance, particularly for those who exercise at home without professional supervision. Also, people hesitate to opt for personal trainers due to expensive consultants and time management issues. Incorrect exercise forms can lead to ineffective workouts and injuries, which can hinder the progress of both normal people and athletes. With recent advancements in computer vision and embedded systems, there is potential to develop a system that guides users in real time to ensure the correct exercise form.

Supplies

Let's start implementing this amazing project, but first, you need a microprocessor like a Raspberry Pi 5 You can also use a Pi 4, but I will recommend using a Pi 5 for better performance and a camera (you can use a webcam or a Pi camera but use one with larger field of view and good quality). Then, a microcontroller Arduino Nano 33 IoT, a laser module, and servo motors like the MG90S.

Step 1: Collect and Prepare a Dataset

Data Collection:

  1. Record videos or images of people performing the target exercises (Pushup, Squat, etc.) using the Raspberry Pi Camera Module 3 (that I am using; you can use other camera too but make sure it is compatible with Pi)
  2. Capture a diverse dataset with variations in angle, lighting, and body types.

Keypoint Extraction:

  1. Use MediaPipe or OpenPose to extract body landmarks for each frame.
  2. Save the extracted landmarks (x, y, visibility) in .npy format for each posture.

Data Preprocessing:

  1. Normalize the landmark coordinates to ensure consistency.
  2. Flatten each sample into a vector of 99 features.
  3. One-hot encode labels for the five postures.

Split Data:

  1. Divide the dataset into 80% training, 10% validation, and 10% testing sets.

I attached an image of how the model will align your body joints with landmarks.



Step 2: Train the Machine Learning Model

Model Architecture:

  1. A Fully Connected Neural Network (FCNN):
  2. Input Layer: 99 features
  3. Hidden Layers: 3 layers with 256, 128, and 64 neurons
  4. Output Layer: 5 neurons (one for each posture) with softmax activation
  5. Use ReLU activation for hidden layers and dropout for overfitting prevention.

Training:

  1. Use Adam optimizer and categorical cross-entropy loss.
  2. Train the model for 50 epochs with early stopping and model checkpoints.

Evaluation:

  1. Monitor training and validation accuracy/loss using TensorBoard.
  2. Save the best-performing model.

I have attached the model training FCNN I made and trained the model from the data of 2500.npy files 500 each for standing, pushups, wall sits, bicep curls, and squats.

Attachments

Step 3: Test and Optimize the Model

Testing:

  1. Evaluate the model on the test set and record metrics like accuracy, precision, recall, and F1-score.

Optimization:

  1. Perform hyperparameter tuning (e.g., learning rate, batch size).
  2. Apply quantization or export the model as .keras for deployment on the Raspberry Pi.

Robustness Testing:

  1. Validate the model with new, unseen data to ensure generalization.

I got a pretty good accuracy and also a validation loss of 0.02 at 13 epochs, which then decreased till 50th epoch.

Step 4: Attaching Servo Motors for Laser Guidance System

Components Required:

  1. Servo Motors (2x):
  2. One for vertical movement (elevation).
  3. One for horizontal movement (rotation).
  4. Common servo models: SG90 or MG995.
  5. Laser Module:
  6. A compact laser pointer or diode module.
  7. Ensure it operates within Arduino’s voltage range (5V).
  8. Mounting Bracket:
  9. A 2-axis servo bracket for mounting the laser module above the motors.
  10. Arduino Nano 33 IoT.
  11. Power Supply:
  12. A 5V power source capable of driving the servos and Arduino.
  13. Physical Assembly:
  14. Secure the horizontal servo motor to a stable base.
  15. Attach the vertical servo motor on top of the horizontal servo using the dual-axis bracket.
  16. Mount the laser module on the top of the vertical servo. Ensure that it is tightly secured and aligned correctly.
  17. Wiring the Components
  18. Horizontal Servo Motor:
  19. Connect the control pin to D9 on the Arduino.
  20. Connect the power and ground pins to the 5V and GND pins on the Arduino.
  21. Vertical Servo Motor:
  22. Connect the control pin to D10 on the Arduino.
  23. Connect the power and ground pins to the 5V and GND pins on the Arduino.
  24. Laser Module:
  25. Connect the power pin to 5V.
  26. Connect the ground pin to GND.



Step 5: Develop Communication Between Devices

Detection of Exercise:

The VisionFit ML model running on the Raspberry Pi detects the user's posture (e.g., "Pushup" detected).

Message Sent to Arduino:

The Raspberry Pi sends a command (e.g., Exercise: Pushup) via Wi-Fi or Serial Communication to the Arduino Nano 33 IoT.

Laser Guidance Activation:

Upon receiving the command, the Arduino:

  1. Activates the laser module.
  2. Adjusts the servo motors to point the laser to the target position for hand placement.


Step 6: Create the Web Interface

Front-End Development:

  1. Use React.js to build a web app with:
  2. Login and registration features using Firebase.
  3. An exercise selection dropdown menu.
  4. Real-time feedback display.

Back-End Development:

  1. Save user-selected exercises and posture data to Firebase.
  2. Implement APIs to fetch and update user progress.

CSS Design:

  1. Use minimalist CSS for a clean and user-friendly interface.


Step 7: Test, Validate, and Document

System Testing:

  1. Test the entire pipeline (data collection, model inference, device communication, and feedback).
  2. Collect feedback from real users to identify improvement areas.

Validation:

  1. Validate the system's accuracy, latency, and robustness under different conditions.

Documentation:

  1. Create a comprehensive report detailing:
  2. Hardware setup
  3. ML model architecture and training process
  4. System integration and testing results


Step 8: Congratulations

So, this is all to make a personal trainer who will guide you or your friend to do exercises with efficiency and no injuries.