PG Department of studies and research in computer science Crafting a Custom mouse with Eye

CHAPTER 1
1.1 INTRODUCTION

• Crafting a Custom Mouse With Eye Tracking

Welcome to the exciting world of Python programming! In this project, we will
be creating an interactive and animated mouse character that features blinking eyes. This
project will help you explore fundamental programming concepts.

• The Crafting Custom Mouse with Eye Tracking

In today’s digital landscape, the mouse is an essential tool for navigating and
interacting with our devices.

uses a web camera with the help of different image processing techniques.

Using figures detection methods for instant Camera access and a user-friendly
interface makes it more easily accessible.

1.2 OBJECTIVES
1. To create a interactive device cost-effectively.

2. To present a hands free interface between computer and human especially for physically disabled
person.

3. To reduce human intervention.

4. To give 100% comfortable environment for disable people.

5. To Design a Customizable and adaptable System for various user needs.

6. To Translate Eye movements into mouse movements.

7. To Implement gesture recognition for click,scroll,and drag actions.

8. To provide assistive technology for seniors and elderly.

9. To comply with accessibility standards and guidelines.

10. To ensure cross-platform compatibility

pg. 1

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

1.3 FEASIBILITY STUDY

The Crafting a Mouse with Eye Tracking is an essential tool for navigating and
interacting with our devices.

uses a web camera with the help of different image processing techniques.

and has been implemented in many programming languages, including Python –

programming.

Technical Feasibility:

1.Image Processing: OpenCV provides robust image processing capabilities, making it feasible to
detect and process visual data.

2. Motor Control: Python libraries like pyautogui,mediapipe,opencv2 enable easy control.

3. Integration: Combining computer vision and motor control is technically feasible.

Resource Feasibility:

1. Cost: Estimated cost for components: $50-$100

2. Time: Project completion time: 2-4 weeks (depending on complexity and developer expertise)

pg. 2

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

CHAPTER 2

2.1 TECHNICAL REQUIREMENT

• Language: Python-Programming

• Tools: PyCharm

• Operating system: Windows 10 and above.

• Minimum hardware requirements:8Gb RAM and 256Gb SSD/HDD

2.2 SYSTEM ANALYSIS

Here's a detailed system analysis for crafting a mouse with Eye Tracking using Python:

System Requirements

Functional Requirements:

1. Image Capture: Capture video feed from camera module.

2. Image Processing: Detect maze walls, obstacles, and navigation paths.

3. Motor Control: Control mouse movement (forward, backward, left, right).

4. Navigation: Navigate through maze using image processing output.

5. Safety Features: Avoid collisions, detect boundaries.

Non-Functional Requirements:

1. Performance: Responsive navigation, minimal latency.

2. Reliability: Consistent performance, fault tolerance.

3. Power Efficiency: Optimize power consumption.

4. Size and Weight: Compact design.

pg. 3

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

CHAPTER 3
3.1 DATA FLOW DIAGRAM
Crafting a Mouse with Eye Tracking using Python

pg. 4

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

CHAPTER 4
4.1 IMPLEMENTATION, CODE, OUTPUT CODE

import cv2
import mediapipe as mp
import pyautogui
cam = cv2.VideoCapture(0)
face_mesh =
mp.solutions.face_mesh.FaceMesh(refine_landmarks=True)
screen_w, screen_h = pyautogui.size()
while True:
_, frame = cam.read()
frame = cv2.flip(frame, 1)
rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
output = face_mesh.process(rgb_frame)
landmark_points = output.multi_face_landmarks
frame_h, frame_w, _ = frame.shape
if landmark_points:
landmarks = landmark_points[0].landmark
for id, landmark in
enumerate(landmarks[474:478]):
x = int(landmark.x * frame_w)
y = int(landmark.y * frame_h)
cv2.circle(frame, (x, y), 3, (0, 255, 0))
if id == 1:
screen_x = screen_w / frame_w * x
screen_y = screen_h / frame_h * y
pyautogui.moveTo(screen_x, screen_y)
left = [landmarks[145], landmarks[159]]

pg. 5

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

for landmark in left:

x = int(landmark.x * frame_w)
y = int(landmark.y * frame_h)
cv2.circle(frame, (x, y), 3, (0, 255,
255))
if (left[0].y - left[1].y) < 0.004:
pyautogui.click()
pyautogui.sleep(1)
cv2.imshow('Eye Controlled Mouse',frame)
cv2.waitKey(1)

//// database of Python protocol////

# Protocol Buffers - Google's data interchange format

# Copyright 2008 Google Inc. All rights reserved.
#
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file or at
#
https://developers.google.com/open-source/licenses/bsd

"""A database of Python protocol buffer generated

symbols.

SymbolDatabase is the MessageFactory for messages

generated at compile time,
and makes it easy to create new instances of a
registered type, given only the
type's protocol buffer symbol name.

Example usage::

pg. 6

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

db = symbol_database.SymbolDatabase()

# Register symbols of interest, from one or multiple

files.
db.RegisterFileDescriptor(my_proto_pb2.DESCRIPTOR)
db.RegisterMessage(my_proto_pb2.MyMessage)

db.RegisterEnumDescriptor(my_proto_pb2.MyEnum.DESCRIPT
OR)

# The database can be used as a MessageFactory, to

generate types based on
# their name:
types = db.GetMessages(['my_proto.proto'])
my_message_instance = types['MyMessage']()

# The database's underlying descriptor pool can be

queried, so it's not
# necessary to know a type's filename to be able to
generate it:
filename =
db.pool.FindFileContainingSymbol('MyMessage')
my_message_instance = db.GetMessages([filename])
['MyMessage']()

# This functionality is also provided directly via a

convenience method:
my_message_instance = db.GetSymbol('MyMessage')()
"""

import warnings

pg. 7

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

from google.protobuf.internal import

api_implementation
from google.protobuf import descriptor_pool
from google.protobuf import message_factory

class SymbolDatabase():
"""A database of Python generated symbols."""

# local cache of registered classes.

_classes = {}

def __init__(self, pool=None):

"""Initializes a new SymbolDatabase."""
self.pool = pool or
descriptor_pool.DescriptorPool()

def GetPrototype(self, descriptor):

warnings.warn('SymbolDatabase.GetPrototype() is
deprecated. Please '
'use
message_factory.GetMessageClass() instead. '
'SymbolDatabase.GetPrototype() will
be removed soon.')
return message_factory.GetMessageClass(descriptor)

def CreatePrototype(self, descriptor):

warnings.warn('Directly call CreatePrototype() is
wrong. Please use '
'message_factory.GetMessageClass()
instead. '

pg. 8

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

'SymbolDatabase.CreatePrototype()
will be removed soon.')
return
message_factory._InternalCreateMessageClass(descriptor
)

def GetMessages(self, files):

warnings.warn('SymbolDatabase.GetMessages() is
deprecated. Please use '

'message_factory.GetMessageClassedForFiles() instead.
'
'SymbolDatabase.GetMessages() will
be removed soon.')
return
message_factory.GetMessageClassedForFiles(files,
self.pool)

def RegisterMessage(self, message):

"""Registers the given message type in the local
database.

Calls to GetSymbol() and GetMessages() will return

messages registered here.

Args:
message:
A :class:`google.protobuf.message.Message` subclass
(or
instance); its descriptor will be registered.

Returns:

pg. 9

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

The provided message.

"""

desc = message.DESCRIPTOR
self._classes[desc] = message
self.RegisterMessageDescriptor(desc)
return message

def RegisterMessageDescriptor(self,
message_descriptor):
"""Registers the given message descriptor in the
local database.

Args:
message_descriptor (Descriptor): the message
descriptor to add.
"""
if api_implementation.Type() == 'python':
# pylint: disable=protected-access
self.pool._AddDescriptor(message_descriptor)

def RegisterEnumDescriptor(self, enum_descriptor):

"""Registers the given enum descriptor in the
local database.

Args:
enum_descriptor (EnumDescriptor): The enum
descriptor to register.

Returns:
EnumDescriptor: The provided descriptor.
"""

pg. 10

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

if api_implementation.Type() == 'python':
# pylint: disable=protected-access
self.pool._AddEnumDescriptor(enum_descriptor)
return enum_descriptor

def RegisterServiceDescriptor(self,
service_descriptor):
"""Registers the given service descriptor in the
local database.

Args:
service_descriptor (ServiceDescriptor): the
service descriptor to
register.
"""
if api_implementation.Type() == 'python':
# pylint: disable=protected-access

self.pool._AddServiceDescriptor(service_descriptor)

def RegisterFileDescriptor(self, file_descriptor):

"""Registers the given file descriptor in the
local database.

Args:
file_descriptor (FileDescriptor): The file
descriptor to register.
"""
if api_implementation.Type() == 'python':
# pylint: disable=protected-access

self.pool._InternalAddFileDescriptor(file_descriptor)

pg. 11

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

def GetSymbol(self, symbol):

"""Tries to find a symbol in the local database.

Currently, this method only returns

message.Message instances, however, if
may be extended in future to support other symbol
types.

Args:
symbol (str): a protocol buffer symbol.

Returns:
A Python class corresponding to the symbol.

Raises:
KeyError: if the symbol could not be found.
"""

return
self._classes[self.pool.FindMessageTypeByName(symbol)]

def GetMessages(self, files):

# TODO: Fix the differences with MessageFactory.
"""Gets all registered messages from a specified
file.

Only messages already created and registered will

be returned; (this is the
case for imported _pb2 modules)
But unlike MessageFactory, this version also
returns already defined nested

pg. 12

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

messages, but does not register any message

extensions.

Args:
files (list[str]): The file names to extract
messages from.

Returns:
A dictionary mapping proto names to the message
classes.

Raises:
KeyError: if a file could not be found.
"""

def _GetAllMessages(desc):
"""Walk a message Descriptor and recursively
yields all message names."""
yield desc
for msg_desc in desc.nested_types:
for nested_desc in _GetAllMessages(msg_desc):
yield nested_desc

result = {}
for file_name in files:
file_desc = self.pool.FindFileByName(file_name)
for msg_desc in
file_desc.message_types_by_name.values():
for desc in _GetAllMessages(msg_desc):
try:
result[desc.full_name] =
self._classes[desc]

pg. 13

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

except KeyError:
# This descriptor has no registered class,
skip it.
pass
return result

_DEFAULT =
SymbolDatabase(pool=descriptor_pool.Default())

def Default():
"""Returns the default SymbolDatabase."""
return _DEFAULT

pg. 14

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting a Custom mouse with Eye

4.2 OUTPUT

pg. 15

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting Custom mouse with Eye

4.2 TESTING

Testing : This is the most frequent step involved, we need to test after
completing each task. We also need to recheck old functionalities once new ones
are developed. We have to make sure the new thing doesn’t affect the older one.

For testing my application I have runned emulator provided Pycharm.

1.Deploy : After getting both applications developed and passing their

testing, the system is ready to be implemented in industries. It is now
ready to be used by society and further development will be
continued lifetime as new technology and ideas arrive.

2. Update and Maintenance : Update and maintenance of applications is an

incremental lifetime step which will be continued as we encounter bugs and
problems.

pg. 16

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting Custom mouse with Eye

CHAPTER 5
5.1 ADVANTAGES
There are several advantages of _“CRAFTING A CUSTOM MOUSE WITH EYE
TRACKING IN PYTHON _”

• Enhanced Accessibility:

Empowering Users: Eye tracking can significantly benefit individuals with

disabilities or limited mobility, allowing them to control their computers without
traditional input devices.

• Innovative User Interfaces:

Creates a an alternative to traditional mouse and keyboard inputs.

• Real-World Applications:

You can easily modify or expand the system, adding features like voice control or
additional sensors.

• Adaptability:

You can easily modify or expand the system, adding features like voice control or
additional sensors.

5.2 LIMITATIONS

pg. 17

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting Custom mouse with Eye

• Technical Complexity:

Learning Curve: For beginners, understanding the integration of hardware and

software can be daunting, requiring a good grasp of both programming and
electronics.

Debugging: Troubleshooting issues in both the software and hardware can be

time-consuming and complex.

• Limited Functionality:

Basic Features: The initial version may lack advanced features found in
commercial products, such as precise calibration, customizable settings, or advanced
gesture recognition.

User Adaptation: Users may need time to adapt to a new input method, and the
learning curve can vary significantly among individuals.

• Maintenance:

Software Updates: As libraries and frameworks evolve, keeping the project up to

date may require ongoing maintenance.

• 5.3 CONCLUSION
In conclusion, the project

"Crafting a Custom Mouse with Eye Tracking in Python “

represents a unique intersection of technology, creativity, and user-centered

design. By leveraging Python’s powerful libraries and tools, such as
Pyautogui, MediaPipe and OpenCV, this project not only demonstrates the
capabilities of modern programming and computer vision but also opens up
new avenues for interaction and accessibility.

5.4FUTURE ENHANCEMENT

pg. 18

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting Custom mouse with Eye

Future enhancements for “CRAFTING A CUSTOM MOUSE WITH EYE

TRACKING IN PYTHON” could include integrating advanced features and technologies to
further streamline processes, improve efficiency, and enhance user experience.

• Advanced Eye Tracking Algorithms

Machine Learning Models: Incorporate machine learning algorithms to
improve the accuracy of gaze estimation and track more nuanced eye movements.

• Cross-Platform Compatibility
Ensure that the software works seamlessly across different operating
systems, such as Windows, macOS, and Linux.
• Accessibility Features Options
Implement features specifically designed for users with disabilities, such
as adjustable dwell times or click thresholds.

BIBLIOGRAPHY

pg. 19

P C JABIN SCIENCE COLLEGE HUBLI

PG Department of studies and research in computer science Crafting Custom mouse with Eye

a. “How to Build crafting a mouse with eye tracking Diksha Jadhav

https://dikshacodes.hashnode.dev/eye-controlled-mouse

b. "Building crafting a mouse with eye tracking” by Programming Hero

c.

d. https://github.com/ProgrammingHero1/e...

e. "How to build a Crafting a Mouse with eye tracking” by Manchala Sreekanth

"
https://medium.com/@manchalasreekanth999/building-an-eye-controlled-
mouse-using-python-and-opencv-f07934a4e15c

pg. 20

P C JABIN SCIENCE COLLEGE HUBLI