TrainERAI - Live Gym Tracker using Artificial

Intelligence
Shezin Saleem Jeremiah Nunes Dr M Aruna

Department of Computing Technology Department of Computing Technology Department of Computing Technology

SRM Institute of Science and SRM Institute of Science and SRM Institute of Science and
Technology Technology Technology

Kattankulathur, India Kattankulathur, India Kattankulathur, India

shezins888@gmail.com Jeremiahnunes5@gmail.com arunam@srmist.edu.in

Abstract— There is a correlation between our workouts with the proper posture by utilizing the most
posture and our physical and emotional wellness. The recent pose estimation methodologies. With simply a
detection of various human postures has been computer and a camera, this concept aims to reduce injuries
approached in a variety of ways. Determine a and enhance human exercise, and also to help people to
patient's resting position, for example, using posture follow an apt fitness plan. The initial step of the project is to
analysis in the medical industry. Image processing use human pose estimation which is a highly applicable
using openCV and mediapipe python library for domain of computer vision. This module in python called
human posture estimation. Analyzing standing and mediapipe helps to identify all the major joints in the human
sitting postures is made easier with an image
body for the processing of the live video for posture
processing-based technique. The benefits of fitness
activities to a person's health are remarkable, but if
estimation. Pose estimate is crucial in addressing issues with
they are done improperly, they may be ineffective and human detection and activity identification. Additionally, it
even detrimental. Exercise blunders happen when aids in the resolution of difficult movement detection
someone doesn't adopt the right posture. This difficulties. After fetching the video from the user we
suggested application makes use of pose estimation to calculate the specific angle of a joint. Later we increase the
identify the user's workout posture and then offers counter for the repetition counter for each set. The counter
specific, tailored advice on what the user can do to only increases only if the exercise is in the correct form. The
correct their posture. A pose estimation module in system also guides the user on the next move with the
python called mediapipe helps to detect all major dialogue box, also it displays the calories burned in each
joints of the human body. Then it calculates the angle exercise. The project as a whole is a complete web
between the joints and increases the counter for each application that helps people to plan a workout, and to
repetition. Pose estimate uses a picture or video of the follow that. The web application is named ‘TrainERAI’, it
subject to determine the precise locations of the has a chatbot named ERAI - which helps with all kinds of
body's major joints. This computer vision technology doubts regarding the application. Also we have a bmi
recognizes human posture in films and displays calculator for finding the bmi score which tells if a user
important regions, such the elbow or shoulder, in the belong to any of the four categories(underweight, healthy,
finished product. Another way to workout at home is
overweight and obese) and suggest a workout planner for
through interactive games. The proposed application
has an interactive boxing game built using pygame,
each category. The best way of workout for all ages is
which helps all age groups to workout and take care through some kind of interactive games. We have an
of their health. Overall the user can track all their interactive boxing game, where the user should hit the target
details on workout on this application, like how much with their fist and if correctly done, score will be
calories are burned and the daily goals, different incremented and can have a scoreboard so all users can
workout planner for each individual. compete with each other. Later on we can add more
interactive games which attract more users. Finally we have
Keywords— Posture Estimation, Artificial Intelligence, a dashboard which displays all the details of a user, it shows
OpenCV, CVZone, Exercise Tracking, Computer Vision, each user's profile, their workout history and the
Detecting posture subscription plan for using these features. We charge each
user an amount of Rs.2000 for a subscription of 100 days,
I. INTRODUCTION and for being consistent with their workout plan and
Exercises like deadlifts, squats, and shoulder presses are achieving their daily goal, user will be getting a maximum
good for maintaining human body fitness, but they may also amount of Rs.15 each day, and for 100 days each user for
be quite dangerous if done incorrectly. Due to the hefty being consistent whole 100 days will be getting a total
weights utilized in these workouts, injuries to the muscles or cashback of Rs.1500. Thereby they can get back the money
ligaments may result. Many people do not maintain the they invested for their workout planner. So we get a profit of
proper posture while completing these exercises on a regular minimum 500 for three months from each user. Then after
basis due to a lack of understanding or instruction. Muscle 30 days users can withdraw the amount of money they made
pain and exhaustion may result from this. By creating a directly to their bank account via UPI or Paytm. As a whole
project that recognizes a user's position while they are it motivates the user for being consistent in their daily
exercising, offers feedback, and proposes modifications as workout and can have a good result.
needed, we are able to assist individuals in executing

Electronic copy available at: https://ssrn.com/abstract=4383182

 II. LITERATURE SURVEY worked with four different exercises, recording training
videos for each, and used geometric heuristic algorithms to
Pose estimation techniques that assess human stances provide personalized feedback on specific exercise
using sensors, movies, and machine learning techniques improvements as well as machine learning algorithms to
have been examined in detail. Several of them are listed automatically determine posture accuracy using only labeled
below. In Paper [1], the authors proposed an approach for input videos. The results of pose estimation were evaluated
the personal health assistant of the software user, also it through the use of human pose keypoints. Limitations are
focuses more on diet rather than the health exercise. It uses they are not live tracking and can’t give calorie and rep
sensors for blood pressure and intensity detection. In this tracking. Paper [8], in this the authors find with the newly
study, we clarified the idea of computer-supported virtual presented geometrical transform kernels, which are simply
reality and its applicability to medical settings. In order to implemented with a convolution layer, the relationships
learn more about computer and Internet usage among between feature maps of joints are recorded. Here the
college students, as well as if these tools are used to proposed framework improves feature learning substantially.
encourage a better way of life, a survey of these students Here also we can’t implement the live video posture
was performed. The findings demonstrate the need for more detection.
efficient methods of educating and raising public awareness
about health issues. The PHA project is a virtually
comprehensive healthcare facility that combines the
majority of necessary healthcare resources into a single III. RESEARCH GAP
application.In Paper [2], the goal was to develop a wearable- ● The main problem which was seen throughout the
based system for power training and tracking. utilizing a survey was that there is no current software out
Samsung Galaxy Gear S2 watch-based workout tracker with there which detects the posture and displays the
a unique user interface (UX) that enables the user to repetition of sets and finds the calorie burned
ascertain the number of repetitions for a certain activity in during each exercise.
the gym, using heart rate to successfully track power and
create a tailored calorie expenditure. This framework ● Another problem, which could be considered a real
enabled it to detect the activity with an average of 96% world problem, is that it can be integrated with the
sensitivity and 96% specificity, distinguish between active current existing gym training application and
and inactive states with 95% accuracy, and count the people can do any exercise in the proper way.
number of repeats for each activity with an accuracy of
>95%. It uses the methodology of data collection using ● Also we can try to add dietary features too for the
Samsung Galaxy Gear S2 and two levels of artificial neural healthy food style of each ones
network and matlab. Paper [3] Estimates By concurrently ● We can completely integrate the project as a web
locating the hand in many depth photos, a 3D hand posture application and host it, so the public can access it.
may be created. They naturally take the form of a multi-task
learning problem, and their effectiveness is assessed using ● None of the existing training applications have
3D hand recommendations made possible by the CNN depth interactive games in it, which helps the users to
image feature map. It uses the techniques of Convolutional burn calories while they are playing any game.
Neural Network(CNN), Resource Description
Framework(RDF) and Region Proposal Network(RPN).
Also To effectively enter motions throughout the data IV. SYSTEM DESIGN
collecting process, you need the right tools. Meanwhile,
Paper [4] outlines the primary motivations for using fitness
trackers, including raising overall fitness levels, lowering
body weight, monitoring heart rate, etc. Utilization of fitness
trackers has increased recently, according to trends. Fitness
tracker is useful in reducing weight, it improves quality
sleep moreover it gives motivation to the users to improve
their general fitness level. A person becomes cautious about
their health.Thus, fitness trackers help in improvement of
general fitness level of the people. The methodology they
used is data collection using survey and Fitness band(google
fit) and smartphone used as fitness tracker device. The
limitations are Insufficient input data to make broad
estimation. Paper [5] demonstrates the rising trend of on-
device computer vision AI fitness teacher applications and
starts to investigate how these technologies are used by
users. able to extract five topics from thematic analysis:
teacher workout, visual feedback, AI discourse, and
computer vision limits. In light of the results and debate, it is Fig 1. Pose estimation and correction block diagram
evident that AI fitness teacher technology may deliver a
compelling and worthwhile user experience. The For the first step of the project, we need to access the
methodology they used are AI fitness instructor mobile system's webcam using the openCV python module. With
applications used : Fitness Ally-AI Workouts Coach , Onyx that we can get the live video recording of the user. Then we
Home Workout. Analytic approach using Thematic Analysis
install the mediapipe module in python, for accessing the
(TA), where analysis of data is in NVivo. Using this
major joints in a human body. As a prebuilt Python package,
combination, to identify patterns. It Provides only
preliminary insights into the experiences of using emerging MediaPipe provides ready-to-use yet configurable Python
AI fitness instructor technologies. Paper [6], the author went solutions. Since we have access to joints of the human body,
through an approach using pose detection library in python, by that we have the coordinates of each joint. With the
which doesn’t have that many features as compared to coordinates with us we can calculate the angle between each
mediapipe. It's also a static posture detection, users should joint with the basic trignometry. Likewise we can calculate
upload a recorded video input and get back an output video the angles between any joints. Now we initialize a threshold
as result. Paper [7], here Electronic
the authorscopy
uses available
the They at: for which we need to increment the counter for the
alsohttps://ssrn.com/abstract=4383182
calculation of repetition of our exercise. Let's consider the correctly then the score is incremented by 10 and the speed
case of biceps curl, where we find the angle between is incremented by1. If the target is not hitted then the speed
shoulder elbow and wrist, then we increase the counter only and score remains the same. It is an interactive game to burn
if the angle varies from 30 to 160 degree, and display it on calories for all age groups and burn out their calories. We
to the screen. We have also included the feature for can extend this feature furthermore, can include more games
guidance of the movement for the user for any exercise. and can include zumba training also in the similar way.
Like what move should be done after we hit any exercise.
Similarly we can calculate and initialize counters for various C. BMI Calculator
parts for exercising. If the user gets bored of the normal
exercises we have an interactive way of workout where the Here we fetch the age, gender, height and weight from the
user can play a game and burn out their calories. Also the user for finding out the BMI score of each individual and
user can find out their BMI with the BMI calculator and the with that score we classify each users into - underweight,
system suggests a workout plan for each user to follow. Also healthy, overweight and obese. Then we suggest different
the user can get back their subscription amount if they can workout plans for each individual according to their BMI
be consistent in their workout and can upto 75% of their score, users can follow this workout and diet plan to be
subscription amount back within 100 days. Overall the user healthy. The system also recommends the calorie intake one
can follow a correct workout plan and can track all their should maintain to be healthy. The BMI is calculated by the
activities too. basic BMI calculating formula, that is

BMI = Weight (kg) / {Height(m)}^2

V. SYSTEM IMPLEMENTATION
and the Calorie intake is calculated using Harris-Benedict
Formula with BMR(basal metabolic rate) - is equivalent to
the amount of energy (in the form of calories) that your
body needs to function if it were to rest for 24 hours, that is

For men: BMR = 66.5 + (13.75 * weight in kg) + (5.003 *

height in cm) - (6.75 * age)

For women: BMR = 655.1 + (9.563 * weight in kg) + (1.850

* height in cm) - (4.676 * age)

D. Dashboard

In the dashboard the user can track their daily workout

history and can view their subscription plans and the current
earning for the day and the earning for the whole month. He
can also view his BMI history and the calorie needed to
maintain his weight and his repart in a graph format to track
his/her progress in the workout. It also shows the time spent
by the user for the exercises. In the user's profile they can
see their current body index and can edit it anytime. Also the
user has an option for withdrawal of the earnings by the
A. Pose Estimation user, only after 30 days the user can withdraw their money
to their specific bank account.
Mediapipe is used to find out the major joints in a human
body and with those joints we find out the angle between E. ERAI - Chatbot
each joint we want to calculate with some basic
trigonometric calculation and then with that angle we create It’s a chatbot designed using kommunicate.io, an AI chatbot
a threshold for which we initialize a counter for the which helps each user regarding any doubts on workout and
repetition counter. Also we guide the user for the next move meal planner inside the web application. If the user is
he/she should follow for the exercise. Also we display the mentally stressed out, then they can seek ERAI’s help for
calories burned during each exercise corresponding to the any queries. It also suggests references online on the web so
repetition of exercise for that particular part. Thereby the that the user can redirect there and find out solutions.
user can ensure that if the exercise they are performing is in Basically its an interface between the system and the user.
the correct form or not. We have mainly three pose
estimation workouts as of now, starting with pull, push and
leg workout. Pull focuses on elbow and back exercises, push
focuses on shoulder, triceps and chest exercises and leg
focuses on all main leg exercises. For now we have included VI. RESULTS
only shoulder, elbow and squat exercises. Different live video of exercises are detected by the system
and the results are shown below:
B. Boxing Game
A. Push Workout:
Pygame, openCV and cvzone is used to build a boxing
game, basically we will be having punching bag target on
screen moving from bottom to up randomly. User’s fist will
be detected using the cvzone library and the target will be
moving from bottom to top, Electronic
and if the user
copyhitsavailable
the targetat: https://ssrn.com/abstract=4383182
 Fig 3. Output video of Leg Workout
For Leg Workout we calculate the angles between the hip,
knee and ankle and finds the angle initializes a threshold for
incrementing the counter

D. Boxing Game:

Fig 2. Output video of Push Workout

For Push Workout we calculate the angles between the hip,
shoulder and elbow and finds the angle initializes a
threshold for incrementing the counter
Fig 4. Output video of Boxing Game
B. Pull Workout: For Boxing game we use pygame, openCV and cvzone for
adding a moving punching bag, and the user should hit the
bag to increase the score, and if the user misses it then score
won't be counted. Ultimately the user should score
maximum to beat other players.

VII. CONCLUSION
Fig 2. Output video of pull workout
This project presents a web application that uses Python's
For pull workout we calculate the angles between the mediapipe module to provide the user real-time posture
shoulder, elbow and wrist and finds the angle initializes a estimate while exercising. The main idea of this feature is to
threshold for incrementing the counter
ensure that the exercises done by the users are in the right
form or not. We also have an interactive game within the
web application which helps the users from all age groups to
C. Leg Workout:
play and workout to stay healthy, in future we can have
more games like this to attract users. Users can also find out
their BMI score, according to that the system finds the
maintenance calorie for each user and with the BMI score
system classify each user under four categories -
underweight, healthy, overweight and obese. For each
separate group it suggests different workout plans. In the
dashboard the user can view their profile which shows the
body index, displays the workout history of each user, also
the earnings made by workout for each user in a graphical
representation. Users can withdraw the earnings made by
then only after 30 days. If the user is not consistent, and
can’t reach upto goal then there will be a small penalty for
not being consistent. By this project the users can stay
healthy in a different way and can be immune to other
diseases.

VIII. REFERENCES
Electronic copy available at: https://ssrn.com/abstract=4383182
1) Ghada A. Al-Mashaqbeh (2013): Computers and e-Health:
Roles and New Applications,
https://ieeexplore.ieee.org/document/6454498
2) Maram Maheedhar, Aman Gaurav, Vivek Jilla, Vijay N Tiwari,
Rangavittal Narayanan, StayFit: A Wearable Application for
Gym based Power Training
https://sci-hub.se/10.1109/EMBC.2016.7592166
3) YuZhanga, Siya Mi, Jianxin Wu, Xin Geng, Simultaneous 3D
hand detection and pose estimation using single depth images
https://www.sciencedirect.com/science/article/abs/pii/
S0167865520303482
4) Durgesh Kumar Durgey, An Analytical study of use and Effects
of Fitness Trackers on Humans
https://www.researchgate.net/publication/
5) Andrew Garbett, Ziedune Degutyte, James Hodge, Arlene
Astell, Towards Understanding People’s Experiences of AI
Computer Vision Fitness Instructor Apps
https://dl.acm.org/doi/fullHtml/10.1145/3461778.3462094
6) Lei Yang, Yingxiang Li, Degui Zeng, Dong Wang, Human
Exercise - Posture Analysis based on Pose Estimation
https://ieeexplore.ieee.org/document/9390870
7) Steven Chen & Richard Yang (2018). Pose Trainer: Correcting
Exercise Posture using Pose Estimation
https://www.researchgate.net/publication/
324759769_Pose_Trainer_Correcting_Exercise_Posture_using_
Pose_Estimation
8) X. Chu, W. Ouyang, H. Li and X. Wang, ”Structured Feature
Learning for Pose Estimation,” 2016 IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), 2016,
https://www.itm-conferences.org/articles/itmconf/ref/2021/05/
itmconf_icacc2021_03031/itmconf_icacc2021_03031.html
9) T. L. Munea, Y. Z. Jembre, H. T. Weldegebriel, L. Chen, C.
Huang and C. Yang, ”The Progress of Human Pose Estimation:
A Survey and Taxonomy of Models Applied in 2D Human Pose
Estimation,”
https://www.semanticscholar.org/paper/The-Progress-of-
Human-Pose-Estimation%3A-A-Survey-and-Munea-Jembre/
746e48e1b6253699403d07a3d2f53cf8ad6bcdc5
10) J. Tompson, A. Jain et al. ”Joint Training of a Convolutional
Network and a Graphical Model for Human Pose Estimation.”
Eprint Arxiv(2014):1799-1807.
https://arxiv.org/abs/1406.2984

Electronic copy available at: https://ssrn.com/abstract=4383182