AIR CANVAS WITH PYTHON - OPENCV AND MEDIAPIPE

Mr. Pradeep1, Mr. Ashwani Shivam2, Mr. Tushar Sharma3, Mr. Rajat Singh4
1
Assistant Professor 2,3,4 Student CSE Department

HMR Institute Of Technology And Management, GGSIPU, Delhi-110036

Abstract

Writing in air has recently emerged as a captivating and complex research area within image processing and pattern
recognition, significantly enhancing human-machine interactions. Researchers are exploring various methods to
reduce processing time and improve recognition accuracy, particularly in the realm of object tracking, which is
essential in computer vision. The rise of faster computers, affordable high-quality cameras, and the demand for
automated video analysis has popularized object tracking systems. This process generally involves three steps:
identifying the object, tracking its movement across frames, and analyzing its behavior, while addressing four main
challenges: object representation, feature selection, identification, and tracking. These algorithms have diverse
applications, including video indexing, autonomous surveillance, and vehicle navigation. This study focuses on
motion-to-text converters for smart wearable devices that enable air writing, utilizing computer vision to track finger
movements to generate text for sending messages and emails, thus offering a valuable communication tool for deaf
individuals and providing a novel way to convey information without traditional writing methods.

I. INTRODUCTION

Digital art uses science and contemporary technology to express artistic ideas. A wide range of artistic expressions,
including visual, aural, and audio-visual works as well as inventive compositions in the fields of literature, painting,
sculpture, music, dance, and architecture, were available prior to the development of digital art. But since digital and
conventional art forms are increasingly entwined, a careful analysis of both is required. While digital art seeks to create
hand motion recognition algorithms for digital writing, traditional writing techniques use pen and paper or blackboards.
Digital art creation is made possible by devices like keyboards, touch-screen surfaces, digital pens, styluses, and
electronic hand gloves. As technology develops, hand motions are recognized using Python programming and machine
learning techniques, improving human-computer interaction (HCI) naturally.

II. EXISTING SYSTEM

Traditional input devices like keyboards and mice have long been the standard for interacting with computers. However,
as technology advances, we're seeing a shift towards more intuitive and efficient input methods. Touchscreens, in
particular, have revolutionized the way we interact with devices like smartphones and tablets. They offer a more natural
and direct way to input information, eliminating the need for physical keyboards and mice.

Voice recognition technology is another significant advancement. By recognizing spoken language, voice recognition
systems allow users to interact with devices hands-free. While voice recognition has become increasingly accurate, it
still has limitations, especially in noisy environments or with specific accents.

The future of input devices promises even more innovative solutions. Emerging technologies like gesture recognition,
eye tracking, and brain-computer interfaces have the potential to further enhance the way we interact with computers.
These technologies could enable us to control devices with natural gestures, eye movements, or even thoughts.

III. PROBLEM STATEMENT

Smartphones overuse-The increasing used of smartphones in children draw their attention from education to the social
media or games. Sometimes excessive used of smartphones lead to depression, causes accident or many times mental
 conditions. There can be probability of life danger due to keeping smartphones very closed.

Paper wastage- Using paper for writing unnecessary things, drawing small things and throwing entire paper led to
conditions of wastage of paper. The fact is many trees need to cut down for making single notebook.

Writing or drawing using air canvas can solve this problem. Also new technology can draw children's attention to
education as it can be used for education. Air canvas uses no paper it requires only some part of memory. A papers
wastage can be not increasing anymore. Safe distance needed while make use of air canvas so it will not affect your
eyes also.

IV. SCOPE

The scope of computer vision, particularly with tools like OpenCV, is vast and influential across various sectors. In
transportation, it plays a key role in autonomous driving through systems like Automated Driver Assistance (ADAS),
which includes features like traffic sign detection, pedestrian detection, and driver fatigue monitoring.

In the medical field, computer vision is essential for analyzing medical images, including mammography and
cardiovascular scans, and for tasks such as automated detection and counting of microorganisms.

Manufacturing also benefits significantly from computer vision, employing techniques like rotation-invariant detection
on conveyor belts to enhance robotic gripping precision.

IV. PROPOSED SYSYTEM

In this proposed framework, we are going to utilize camera and the screen for the reading inputs and displaying
outputs. We are using our hand fingers to drawing required shapes on the output screen. We have to be on safe
distance where our hand can be fully visible in camera hence it is reading our input by recognizing movements of our
fingers tip. Some other hand sign for selecting shapes and draw going to use as per given in used modules and
libraries.

V. APPLICATIONS

Thanks to the numerous techniques offered by OpenCV, Python may be used to rapidly analyze images and videos and
extract valuable information from them. Additional common applications include image processing:

Image Processing:

Images can be processed and interpreted using OpenCV in a number of ways, including changing their color or shape or
collecting crucial information from the provided image and transferring it to a new image.

Face Detection:

Face detection is accomplished by using Haar-Cascade Classifiers, either from locally stored images or videos or from
live webcam streaming.

Face Recognition:

OpenCV was used to generate bounding boxes (rectangles) and then train the model using machine learning techniques
in order to recognize faces in the movies.

Object Detection:

To detect moving or stationary objects in pictures and videos, OpenCV and YOLO, an object identification technique,
can be utilized.
VI. RESULTS/SNAPSHOTS
 VII. STEPS FOR USE

Step 1: Launch the app on a portable device with a good camera, or, if necessary, an external camera attached.

Step 2: Place yourself far enough away from the camera so that the camera on your device can clearly see your hand and
fingers.

Step 3: Learn the signs for the hand motions needed to carry out tasks like drawing the necessary shape or choosing or
switching tools.

Step 4: Choose the necessary shape from the device's screen and begin sketching.

VIII. FUTURE SCOPE

The goal of the field of computer vision is to make it possible for computers to comprehend and interpret pictures and
movies. It has made great strides over time, frequently surpassing human pattern recognition, especially in the medical
field. With more research likely to expand its capabilities, computer vision appears to have a bright future. Innovative
applications will result from systems' increased ability to recognize features in images and integrate with other AI
technologies as they are easier to train. For example, image captioning, which combines natural language creation and
computer vision, may help people with visual impairments.

Furthermore, the development of virtual and augmented reality (VR and AR), which has attracted a lot of interest from
many tech companies, is directly related to computer vision. The launch of numerous innovative goods reflects this
expanding emphasis. All things considered, computer vision's importance in technology is expected to grow further
given its ability to support developments like artificial general intelligence (AGI) and artificial superintelligence (ASI).

IX. CONCLUSION

Especially in creative and educational environments, the Air Canvas project offers a novel way to do rid of the problems
caused by conventional input devices like the mouse. This system provides a more user-friendly and hands-free method of
drawing, presenting, or interacting with digital content by utilizing the MediaPipe library for effective hand tracking.
Because MediaPipe can precisely identify hand locations, it eliminates the need for intricate image processing, which makes
the device simple to use and efficient.

This method improves accessibility, creativity, and engagement in a variety of applications, such as education, painting, and
sign language recognition, while also lessening dependency on physical hardware. Furthermore, the Air Canvas system's
adaptability makes it a useful starting point for upcoming advancements in hand tracking, including enhanced sign language
detection, gesture-based navigation, and virtual mouse control.

In conclusion, the Air Canvas system offers a viable and effective substitute for conventional interaction techniques, with a
great deal of room to grow into new technological and accessibility domains in the future.

X. ACKNOWLEDGMENTS

To everyone who has helped us during this endeavor, we would like to sincerely thank you. We are especially appreciative
to the people who helped us overcome obstacles and maintain focus on our goals by offering priceless support at every turn.

We would especially want to thank our friends and coworkers, whose constant encouragement and support have been
crucial to the accomplishment of this project. Their unwavering support has fortified our determination and given us the
self-assurance to execute and finish the assignment without any problems.
XI. REFERENCE

1. OpenCV Documentation:

• OpenCV. (n.d.). OpenCV Documentation. Retrieved from https://docs.opencv.org/

2. Gesture Recognition Using OpenCV:

• H. K. (2020). Gesture Recognition with OpenCV. Learn OpenCV. Retrieved from

3. Python for Computer Vision:

• Rosebrock, A. Python for Computer Vision with OpenCV and Deep Learning. PyImageSearch. Retrieved from
https://www.pyimagesearch.com/

4. MediaPipe:

• Google. (2023). MediaPipe: Cross-Platform Framework for Building Pipelines to Process Video, Audio, and Other
Multimedia Types. Retrieved from https://mediapipe.dev/

5. Hand Tracking Example:

• Hu, Y. (2019). Hand Tracking Using OpenCV. GitHub Repository. Retrieved from
https://github.com/HandTracking

6. Real-Time Hand Gesture Recognition:

• Choudhury, A., & Kumar, A. (2019). Real-Time Hand Gesture Recognition Using OpenCV. International
Journal of Computer Applications, 975, 8887. doi:10.5120/ijca2019918980

7. Community Engagement:

• OpenCV. (n.d.). OpenCV Forum. Retrieved from https://forum.opencv.org/