Fingerprint Recognition System
Fingerprint Recognition System
Fingerprint Recognition System
Submitted By
SARVESH GAONKADKAR
OM GHAG
TEJAS KUMBHAR
ANURAG KUMBHARE
University of Mumbai
2024-25
CERTIFICATE OF APPROVAL
For
Project Synopsis
SARVESH GAONKADKAR
OM GHAG
TEJAS KUMBHAR
ANURAG KUMBHARE
Examiner 1 Examiner 2
Principal
Declaration
We declare that this written submission represents our ideas in our own words and where
others' ideas or words have been included, we have adequately cited and referenced the original
sources. We also declare that we have adhered to all principles of academic honesty and integrity and
have not misrepresented or fabricated or falsified any idea/data/fact/source in our submission. We
understand that any violation of the above will be cause for disciplinary action by the Institute and can
also evoke penal action from the sources which have thus not been properly cited or from whom proper
permission has not been taken when needed.
2. Om Ghag 21102B0002
Date:
Acknowledgements
This Project wouldn’t have been possible without the support, assistance, and guidance of a
number of people whom we would like to express our gratitude to. First, we would like to convey our
gratitude and regards to our mentor Prof. Sneha Annappanavar for guiding us with her constructive and
valuable feedback and for her time and efforts. It was a great privilege to work and study under her
guidance.
We would like to extend our heartfelt thanks to our Head of Department, Dr. Sachin Bojewar,
for overseeing this initiative which will in turn provide every Vidyalankar student a distinctive competitive
edge over others.
We appreciate everyone who spared time from their busy schedules and participated in the
survey. Lastly, we are extremely grateful to all those who have contributed and shared their useful
insights throughout the entire process and helped us acquire the right direction during this research
project.
Table of Contents
1. Abstract
2. Introduction
4. Literature Survey
5. Problem Statement
6. Scope
7. Proposed System
8. Methodology
9. Analysis
10.Design
12.References
Abstract
Because of the advancements in the deep learning and computer vision, Convolutional Neural
Networks (CNNs), has become a key framework for image related tasks including fingerprint
recognition system. On one hand, fingerprints have been matched with traditional techniques such as
Scale Invariant Feature Transform (SIFT) and Oriented FAST and Rotated BRIEF (ORB) along with
CNNs. However, previous studies have demonstrated that CNNs are not well suited to working with
large and complex datasets. SIFT, although accurate, is computationally expensive whereas ORB is
fast, but precise.
For fingerprint recognition tasks of fingerprint matching, Siamese Neural Networks (SNNs) have been
employed to overcome these challenges. SNNs are unlike traditional methods, which learn the
similarity among input fingerprints, and hence are very good for verification and matching tasks.
Fingerprint pattern recognition is shown to be improved with SNNs, and SNNs are especially capable of
distinguishing genuine from false fingerprints, when there is noise or partial prints. For mobile security
systems with limited computational resources, high efficiency is a must, and this is important.
Fingerprint recognition systems based on SNNs can provide the advantages of the pairwise
comparison rather than global classification, and the more accurate and reliable fingerprint matching.
While SNNs have been designed for local feature extraction and matching, their lightweight architecture
enables them to be deployed on mobile security applications, which demand resource efficiency.
In this paper we evaluate the application of SNNs in mobile fingerprint recognition systems, from a
comprehensive point of view, in order to understand the mobile security context in which SNNs can be
applied. Results indicate that for real time biometric authentication on mobile devices, SNNs offer an
excellent balance between accuracy and computational efficiency and are thus a good solution.
Through this study we validate the potential of SNNs to improve the performance and security of mobile
based fingerprint recognition systems, at a time when there is growing demand for secure and efficient
mobile biometric solutions.
Introduction
Fingerprint recognition systems have largely relied on Convolutional Neural Networks (CNNs) for
feature extraction as well as pattern recognition tasks. Strong performance across multiple biometric
applications has been made possible due to their ability to capture local spatial hierarchies within
images. CNNs, however, suffer in computational efficiency and the handling of global contextual
information — both of which are important in mobile security environments with limited resources.
In order to alleviate these problems fingerprint recognition employs the techniques of Siamese Neural
Networks (SNNs) especially in mobile security applications. Using a twin network architecture, which
learns similarity metrics between two inputs, SNNs are particularly well suited to tasks like fingerprint
matching and verification. SNNs can compare pairs of fingerprint images to check if the produced
images are from the same individual, with the additional advantage of having greater accuracy and
robustness in a biometric environment that poses a challenge.
As SNNs are compute efficient and are naturally focused on pairwise comparisons, they are especially
advantageous for mobile security, where fingerprint verification requires such comparisons. Unlike
traditional CNNs, SNNs are specifically designed to compare individual image pairs, and results
indicate that they are robust to fingerprints that are noisy or partially captured. SNNs also have the
appropriate resource constrained nature of mobile platforms, where computational demands can be
balanced with accuracy.
In this research, we demonstrate how the architecture of SNNs addresses the unique challenges of
mobile security and apply them to mobile fingerprint recognition systems. The combination of high
accuracy and speed makes SNNs ideal for mobile device biometric authentication systems, allowing for
fingerprint recognition systems to take advantage of SNNs to achieve both.
An SNN is used in the project that can compute similarity score between fingerprints images to do
correct identification. Stored biometric data will be protected by AES encryption, and TFLite will
optimize the model for mobile performance.
Objectives:
- Develop an SNN model: For fingerprint recognition, we create and train a Siamese Neural
Network addressing challenges like lighting, rotation, and partial fingerprints.
- Ensure data security through AES encryption: Store fingerprint data based on AES encryption
to control the unauthorized access.
- Optimize the model for mobile platforms using TensorFlow Lite: Then, we convert and optimize
the SNN model for efficient mobile deployment using model quantization and GPU delegation.
- Integrate augmentation techniques: It is the focus to augment training with the aim of improving
the model’s robustness to variations in the images of fingerprints.
- Enable real-time fingerprint matching: Create the system for quick comparisons on the platform
using mobile hardware resources in order to authenticate users seamlessly.
- Deploy the system on mobile devices: Install the fingerprint recognition system on as a
fingerprint app on a mobile application where the users can enrol and identify fingerprints
securely.
Literature Survey
Fingerprint recognition systems have evolved from conventional Convolutional Neural Networks (CNNs)
to more complex Siamese Neural Networks (SNNs). The purpose of these advances is to drive
biometric systems to higher accuracy, greater efficiency, and increased robustness. In this review, we
provide an overview of key SNNs and CNNs developments, as applied to fingerprint recognition in
mobile applications.
Convolutional Neural Networks (CNNs): CNNs have become central to advances in image
recognition, from fingerprint recognition to image recognition, because they are capable of recognizing
spatial patterns and hierarchies of images. Major contributions in this area are:
- LeNet-5 (1998): The first CNN model proposed by LeCun et al, among the first to demonstrate
the power of convolutional layers to solve image classification problems, is applied to the image
of handwritten digits. To use CNNs for fingerprint recognition, we were able to show that
feature extraction from images can be done effectively by this model.
- AlexNet (2012): The most recent piece of technology introduced in the world of image
classification is AlexNet by Krizhevsky et al that brought a lot of improvements using deep
layers, ReLU and drop out. In large scale image datasets, it proved successful, and it has
established itself as an important milestone in CNN development, but beyond biometric tasks
such as fingerprint recognition.
- VGGNet (2014): In fact, what Simonyan and Zisserman introduced was VGGNet, a deeper
architectures and smaller convolutional filters combo to boost feature extraction power. This
approach requires additional computational requirements that make it unsuitable for
deployment in mobile environments but is very effective in capturing fine image details.
- ResNet (2015): But to deal with the vanishing gradient problem in deep networks, he et al.
introduced the residual connections as ResNet. This innovation of deep learning allowed deep
networks to be effectively trained, and it seemed to be well suited to fingerprint recognition
tasks that require extracting a more complex feature and high precision.
Despite the success of CNNs in feature extraction, their limitations in handling pairwise comparison
tasks, essential in Though CNNs have excelled in feature extraction, they remain constrained in the
pairwise comparison tasks that are key to fingerprint matching and are consequently employed for
fingerprint matching using Siamese Neural Networks (SNNs).
Siamese Neural Networks (SNNs): The usefulness of SNNs lies in the fact that they are specially
made for tasks where we need to measure the similarity between two inputs, as in biometric matching
systems such as fingerprint recognition.
- SNNs for Signature Verification (1994): SNNs are introduced by Bromley et al. for the task of
signature verification and show that the SNN is able to learn a similarity metric between two
inputs. This approach also set the stage for the use of SNNs to other biometric tasks including
fingerprint recognition.
- Face Verification with SNNs (2015): Koch et al. further validated the application of SNNs to
face verification, as this task requires pairwise comparisons, a requirement naturally met in
biometric authentication where two image samples are compared.
- Fingerprint Recognition with SNNs (2017): Since then, SNNs have been used on fingerprint
recognition tasks where the model learns to compare two fingerprint images and learn a
similarity function. A robust matching of fingerprints is provided by this method, in the case
where the prints are noisy or incomplete.
The selection of SNNs’ architecture through which pairwise comparison is performed is particularly well
suited for mobile fingerprint recognition systems due to the fact that new fingerprints can be compared
against stored biometric templates in an efficient manner.
- Pairwise Matching: The twin architecture of SNNs allows the system to achieve high accuracy
in matching tasks by comparing pairs of fingerprints, by attenuating the differences between
fingerprint images.
- Efficiency: A variety of SNNs are optimized for mobile platforms, which provides efficient
performance without sacrificing accuracy, an important property for mobile biometric systems.
- Robustness: The advantage of SNNs is that they are able to handle noisy or partial fingerprint
data, increasing system robustness in real world mobile applications where fingerprint scans
may not be the best, and making it easier to design for such situations.
Problem Statement
Problem Statement:
With mobile devices heavily playing a role in one’s personal and work life, secure and facile
authentication systems have been an absolute necessity. Traditional password-based authentication
methods are becoming increasingly insufficient, as password breaches, phishing attacks, and negligent
password management increasingly threaten them. So, in this context, fingerprint recognition becomes
more secure and user friendly. But making fingerprint recognition systems effective on mobile devices
is not a trivial problem.
Scope
The focus of this work is building and deploying a Siamese Neural Network (SNN) based fingerprint
recognition system that is secure, efficient, and accurate for mobile devices. It covers several important
areas like:
- Development of the SNN Model for Fingerprint Recognition: It will be the focus of the project to
build a strong Siamese Neural Network that can accurately compare fingerprint images.
Different augmentation techniques are used to train the model to handle variations in fingerprint
presentation like difference in orientation, pressure as well as partial captures. It is to make
sure the system is still able to separate normal users from those allowed.
- Security of Fingerprint Data through AES Encryption: The system will ensure that encrypted
biometric data is used and that it will be implemented during the profile registration process
using AES. This is this critical measure that protects user’s fingerprint data from unauthorized
access and if data storage is compromised then also biometric information does not come out
from the safe unless proper decryption key is not present.
- Optimization for Mobile Devices using TensorFlow Lite: Since the resource constraint of mobile
platforms, the project will make use of TensorFlow Lite to convert and optimize the SNN model
to reduce the time consumption in mobile devices. The focus will be on reducing the
computational overhead and making the system work smoothly within the limited resources
such as processing power and memory provided by such devices as smartphones and tablets.
- Real-Time Fingerprint Matching for User Authentication: A real time fingerprint matching
system will be designed for real time fingerprint matching with a requirement of low latency to
provide a quick and smooth user authentication. The model provides a means to use mobile
hardware resources effectively and perform fast matching between the stored fingerprint and
the one during login attempts for usability.
- Integration with Real-World Mobile Applications: The last objective is to deploy the fingerprint
recognition system on Android devices. Then the system will be integrated with mobile app,
where users can register their fingerprints securely, store them safely and use them for
authentication during actions like unlocking their device or authorising transactions.
- Augmentation and Robustness: During the training phase, multiple image augmentation
techniques will be used to increase robustness to the system. The model generalizes better to
real world conditions, and it can effectively handle variations in the fingerprint quality and
presentation.
Proposed System
For fingerprint recognition in mobile security, the proposed system implements Siamese Neural
Networks (SNNs). In contrast to standard biometric recognition techniques, the SNN architecture is
specially developed to perform pairwise comparisons, thus being a good candidate for fingerprint
matching. The goal of this system is to produce a highly efficient and accurate fingerprint recognition
model that can be deployed on mobile devices and run in real-time with secure data processing.
System Overview:
An SNN is used as the core of the system to compare two fingerprint images to identify their similarity.
With architecture optimized for small/mobile footprint, great computational overhead is minimized, and
maximum battery is attained. The design consists of a chain of convolutional layers which extract
salient features from input fingerprint images, and then dense layers that map each image to a compact
feature embedding. These embeddings are used for comparing fingerprint during authentication. The
user takes their fingerprint at the time of initial profile registration, then securely stores It on their mobile
device and uses it to generate the user’s biometric profile. In this process, the background adds
fingerprint augmentation to the fingerprint image (rotations, translations, and flips etc.) to train the
model and make it generalize. In this phase, the fingerprint data are stored secured by AES encryption
so that it is secure from sensitive information while being stored. During login attempts, the user’s input
fingerprint is then compared to the stored fingerprint data. After this initial phase the system processes
fingerprint inputs in real time without storing further fingerprint data. This is done to increase security
because encryption is handled by the mobile device’s operating system (OS) and sensitive data is
stored.
Key Components:
- Lightweight Architecture: Traditional CNN based models require much more computational
power, and especially the SNN architecture, when optimized for mobile, demands far less. Due
to the efficiency required for use on smartphones, it is highly suitable for use in such a form.
- Secure Fingerprint Storage: During the initial registration, the user’s fingerprints are stored in
the system using AES encrypted, so that sensitive biometric data is protected. This feature
enables system to customize the fingerprint recognition process for each user. During this
phase augmentation is used to add an extra accuracy of the model.
- Scalability: The SNN model is lightweight and once registered does not require the permanent
storage of new fingerprint templates. It’s easy to deploy across different mobile platforms,
irrespective of the hardware specifications. It is designed to scale, with modifications to the
underlying architecture needed only for a small amount.
- Battery Efficiency: During inference, power consumption is minimized for mobile application,
which is critical, once converted to TFLite, the SNN model. The system prevents fingerprint
recognition from using up the device’s battery with excessive energy.
Methodology
A structured approach to developing the Siamese Neural Network (SNN) for mobile fingerprint
recognition is presented, beginning with real time data collection, preprocessing, model training,
encryption, mobile deployment and evaluation of performance.
Fingerprint data is collected in real time when the user first registers their identity and credentials on the
mobile device in this system. When this registration phase takes place, the fingerprint image is
captured and safely stored by AES encryption. The foundation of the user’s biometric profile is based
on this fingerprint data. The preprocessing steps are essential to maintain consistency and quality in
the input data before training the model:
- Image Resizing: We resize all fingerprint images into a fixed size of 96x96 pixels. The
standardization that comes from this size also reduces computational resources and speeds up
processing time because this size is small enough. Smaller images, as the system is more
efficient on mobile devices, most of them have limited processing power.
- Normalization: Once resized, pixel values are scaled between 0 and 1. This last step makes
the model converge faster in training and adds our model’s ability to generalize under a variety
of lighting conditions, or fingerprint quality.
- Data Augmentation: During training we use data augmentation techniques in order to increase
the robustness of the model. Rotation, flipping, and translation are techniques such as these
which aid a SNN to be able to recognize fingerprints in different orientations or in different
conditions. It is important in dealing with the variability inherent in real world applications.
A convolutional layer, Max pooling layer and dense layer are all used in building the SNN model. The
architecture is designed to extract distinctive features from fingerprint images and generate
embeddings for comparison.
- Training Objective: A contrastive loss function is used to train the model, minimizing the
distance between embeddings of matching fingerprints and maximizing the distance between
nonmatching pairs. For the pairwise comparison tasks, this is an ideal approach, and the SNN
also accurately distinguishes genuine and impostor fingerprints during the authentication
process.
- Hyperparameter Tuning: We fine tune these key hyperparameters, learning rate, batch size
and number of epochs in order to find the optimal model performance. In order to avoid
overfitting, we employ early stopping, allowing the model to generalize well among new unseen
data.
- Encryption and Secure Storage: Biometric data is stored securely with our fingerprint data
stored using AES encryption at the time of registration. During a login attempt, the encrypted
fingerprint data is used for future comparison. Once the fingerprint has been matched against
the user’s input fingerprint, the system executes according to the mobile device’s security
protocols of data handling validating both data integrity and privacy.
- Training Objective: A contrastive loss function is used to train the model, minimizing the
distance between embeddings of matching fingerprints and maximizing the distance between
nonmatching pairs. For the pairwise comparison tasks, this is an ideal approach, and the SNN
also accurately distinguishes genuine and impostor fingerprints during the authentication
process.
- Hyperparameter Tuning: The model performance is fine-tuned on key hyper parameters such
as learning rate, batch size and the number of epochs. To prevent overfitting, we perform early
stopping, so that once our model sees new, unseen data, it generalizes well.
- Encryption and Secure Storage: In order to guarantee sensitive biometric information’s
security, the user’s fingerprint data is stored at the time of registration, using AES encryption.
During a login attempt, the encrypted fingerprint data is used for future comparison. A
comparison between the pre-stored fingerprint and the user input fingerprint is done, and after
the comparison, the data is processed in line with the mobile device’s security protocols to
match data integrity and privacy.
Analysis
Figure A: SNN (Subtract) Model Train and Test Accuracy Across 100 Epochs:
Here the comparison of train and test accuracy for a Spiking Neural Network (SNN) model with subtract
operation over 100 epochs is plotted in this graph.
Key Observations:
Analysis:
- The blue line represents the train accuracy of the SNN (subtract) model for which the accuracy
begins at approximately 70% and increases up to a little above 96% at the end of training
indicating effective learning.
- The orange line represents the test accuracy, starting at a comparable value, and increasing
more quickly to almost 98% after the 20th epoch and then staying at a stable level.
Generalization to unseen data shows strong evidence of minimal overfitting and this trend.
Conclusion: The high accuracy values of the two curves and the closeness of its curves imply that
the SNN (subtract) model performs well in learning without losing too much in the generalization
sense.
Figure B: The Matching Rate Comparison of Models:
The matching rates of three models are compared in a verification or classification task and this bar
graph compares them.
Key Observations:
- Y-axis (Matching Rate %): It represents the accuracy of each model to identify or match data.
- X-axis (Models): It compares three different approaches.
Analysis:
- Traditional Minutiae-based (Gray Bar): It achieves a matching rate of about 60% which is
relatively low.
- CNN-based (Orange Bar): It demonstrates a matching rate of around 85% which is a better
accuracy compared with the traditional method.
- Siamese Neural Networks (SNNs) (Blue Bar): It achieves matching rates close to 100% and
demonstrates the best performance of the models.
Conclusion: The best performing models are Siamese Neural Networks (SNNs), followed by CNN
based models, and traditional minutiae-based models perform the worst.
The storage requirements for each of the three models are shown in this bar graph.
Key Observations:
- Y-axis (Storage Requirement in MB): It is the memory required per model.
- X-axis (Models): The same three models are then compared.
Analysis:
- Traditional Minutiae-based (Gray Bar): It takes about 1 MB of storage, making it the least
memory demanding option.
- CNN-based (Orange Bar): This means that the storage requirements are significantly higher
than that.
- Siamese Neural Networks (SNNs) (Blue Bar): Strikes the balance between minutiae-based
model and CNN based model, using around 3 MB.
Conclusion: SNNs use the least storage followed by the traditional minutiae-based model and CNN
based models with the most storage. We compare these results and find that SNNs offer the best
compromise between accuracy and memory efficiency.
Figure D: Speed
Comparison of Models:
Below is a graph depicting the time (in milliseconds) taken per match by each model.
Key Observations:
- Y-axis (Time per Match in ms): Indicates how long each model takes to perform a match.
- X-axis (Models): Compared to the following models:
Analysis:
- Traditional Minutiae-based Model: The time of shows the highest at around 40 ms which
means it is one of the slowest to do among the options.
- CNN-based Model: One that is faster than the minutiae based (approximately 30 ms per
match).
- Siamese Neural Network (SNN): With a time per match below 10 ms, the fastest model.
Conclusion: Figures show that Traditional Minutia based models are the slowest and SNNs deliver the
best performance in terms of speed and hence suitable for time critical applications.
Key Observations:
Analysis:
On a logarithmic scale, this graph compares the computational complexity of Traditional Minutiae
based, CNN based, and SNN based fingerprint matching models using the number of Floating-Point
Operations per Second (FLOPs).
Key Observations:
Analysis:
- Traditional Minutiae-based Model (Gray Bar): Has the lowest FLOPs, around 10 6. It is found
that this method demands much less computational resources than the other models.
- CNN-based Model (Orange Bar): The FLOPs are displayed at approximately 10 11 This
suggests that CNN based models are very computationally expensive and require much higher
number of operations to do fingerprint matching.
- Siamese Neural Network (SNN)-based Model (Blue Bar): Shows a FLOPs value around 10 10
Still orders of magnitude above traditional minutiae based approaches, , which is slightly lower
than CNN based models.
Conclusion: Previous comparisons indicate that the least computationally demanding (but likely less
accurate) approach to face recognition is the Traditional Minutiae-based method. On the contrary, CNN
based models need the most computational power and yield the best accuracy. With computational
complexity of lower than CNNs and superior performance (high accuracy and speed), SNN based
models are a good candidate for fingerprint matching applications.
Figure G: The relationship between time per matching and similarity score:
The relationship between two important variables in the matching process is illustrated with this scatter
plot.
Key Observations:
- X-axis (Time per Matching in seconds): Usually indicates the time needed to do a matching
operation, e.g. nearest neighbours or similarity matching algorithm.
- Y-axis (Similarity Score per Matching): It is a measure of similarity between two entities in
terms of matching process, from score of 0 to 1, where 1 is the maximal similarity.
Analysis:
- The plot most of the data points cluster in the top left area, around a similarity score of 1 and
time per matching around 0.06 seconds. This means that a large number of matching
operations are both very accurate (similarity close to 1) and very fast (small time per matching).
- Lower similarity scores (between 0.5 and 0.9) with different matching time (between 0.065 and
0.085 seconds) are shown in a few outlier points. These cases correspond to situations where
matching took longer or produced less similarity.
Conclusion: By plotting the trade-off between matching speed and similarity score, we show that faster
matching times produce more accurate results in general. But the outliers indicate that some matches
will take longer and provide less accurate results.
Design
1. Data Preprocessing Flowchart:
This flowchart describes the critical steps for the preparation of a biometric input data for
processing within the model.
- Start (Raw Biometric Input): Biometric data, specifically, fingerprint images are
captured at the raw level, and after that the process starts.
- Input Formatting (Resize to 96 x 96 Pixels): The raw input images are then resized to a
common dimension of 96x96 pixels to standardize the image size to help ease efficient
processing. Along with this, the computational load is reduced and the resizing
guarantees consistency in the data set.
- Normalization (Scale Values to Range (0,1)): All images are normalized to the range
from 0 to 1 in terms of pixel values. The scaling improves the training efficiency and
achieves higher consistency of the model input to learn from the data.
- Data Augmentation (Apply Flipping, Rotation, Translation): Different augmentation
techniques are used to change the images, such as flipping, rotating and translating in
order to enlarge the training dataset. In this approach, the model generalizes better
and is more robust to the variation of fingerprint presentation.
- Output (Pre-processed Biometric Data): To a certain extent, the biometric data is
rendered suitable for model input by following these preprocessing steps; therefore,
this biometric data can be further processed and analysed.
2. Model Architecture:
The architecture of the Siamese Neural Network used for fingerprint recognition is detailed in
this flowchart which showcases its multistage design.
- Two Inputs (Fingerprint 1 and Fingerprint 2): An architecture is presented, which starts
with two different inputs, each corresponding to a fingerprint. First, the user’s profile
(previously registered fingerprint) provides one input, and the second input is captured
when the user attempts to login. The setup allows us to compare the biometric data.
- Feature Model: Each fingerprint is processed by a specialized feature extraction model
that extracts salient features necessary for good comparison.
- Subtraction Layer: A subtraction layer turns into feature representations of both inputs,
and they are compared using it. That allows the model to work with the differences in
the fingerprints.
- Convolution and Pooling: We process the resultant differences through a series of
convolutional layers followed by pooling operations. At this stage, we need to extract
hierarchical features whilst simultaneously keeping dimensionality low, in order to
enable the model to find relevant patterns.
- Flatten: The flattened processed features are then fed to feed to the following dense
layers.
- Dense 64, Dense 1: It is flattened vector through fully connected (dense) layers. The
first dense layer with 64 neurons is intended to capture complex feature interactions,
and the second, dense layer contains a single neuron which outputs a single prediction
score.
- Sigmoid: Finally, a final dense layer feeds to an application of a sigmoid activation
function to transform raw output into a probability score indicating the probability of
match.
- Output (1: final output binary (Same, 0: Not Same): The final output is binary, a value
of 1 means match (inputs belong to the same person), a value 0 means do not match
(inputs do NOT belong to the same person).
3. Security Architecture:
The secure fingerprint data processing flow using a Siamese Neural Network (SNN) for
fingerprint matching in an Android device context is shown in this diagram. It highlights the key
security items that protect biometric data from the time of capture to the time of processing, and
ultimately to the time of storage.
- Biometric Data Capture: The process of security begins with capturing biometric data
which is fingerprints via the device sensors. This initial stage is very important, so we
have to make sure that the input data is handled securely.
- Data Encryption (AES-256): The biometric info is captured and after that the biometric
information goes through AES-256 encryption, which is considered one of the most
robust encryptions. This step in fact protects from exposure of sensitive biometric data
raw form transmission, storage, or via unauthorized access.
- Secure Storage (Android Keystore): And finally, in the Android Keystore is the
encrypted biometric data and the TFLite model used for inference. This component
ensures that cryptographic keys stay contained within the device, so that defense
against unauthorized tampering and access to encrypted fingerprint data and the
machine learning model used to learn who is who is strengthened.
- BiometricPrompt API Authentication: Before further processing it relies on the
BiometricPrompt API to check for user authentication. This API makes it easy for a
secure authentication flow using the biometric capabilities of the device, e.g.,
fingerprint or facial recognition, to be used only with those who are allowed to activate
the SNN model for fingerprint matching.
- Access SNN Model for Inference: The system retrieves the Siamese Neural Network
(SNN) model for inference upon successful authentication. The fingerprint comparison
process with this model is highlighted with the secure TensorFlow Lite model and the
input fingerprint is run through it to compare the stored biometric data to make a
similarity determination and produce a matching score.
- Output (Encrypted Result Storage): The SNN inference produces a result, i.e. does it
match or not, which is securely stored. The matching decision is encrypted and stored
in a storage location that limits access by only authorized personnel, leaving the
confidentiality of the matching decision undisturbed. It protects the results from
possible data breach or unauthorised exploitation.
- Mobile Device: Android smartphones or tablets with built in fingerprint sensor, minimum 2GB
RAM, running Android 8.0 or later.
- PC for Development: Processor: At least 2.5 GHz Intel i5 or equivalent. RAM: Training and
preprocessing tasks take up 8GB or higher.
- Storage: Datasets and models require at least 100GB available. GPU: Accelerated model
training recommended NVIDIA GPU.
Software Requirements:
- Operating Systems:
Development: Windows 10, macOS, or Linux.
Deployment: If you have Android, version 8.0 or later.
- Development Tools: Python: For SNN implementation and the model training.
- TensorFlow & Keras: For building, training and optimizing of the SNN model.
- Android Studio: It can be used to develop the mobile application.
- AES Encryption Tools: AES encryption is implemented using PyCryptodome.
- Libraries and Dependencies: OpenCV, NumPy, Pandas, TensorFlow/Keras, TFLite Converter.
- Version Control and Collaboration Tools:
Git: For version control.
GitHub/GitLab: It is for cloud-based code repositories.
References
1. https://www.ijert.org/research/finger-print-matching-algorithm-for-android-
IJERTV2IS101166.pdf
2. https://www.researchgate.net/publication/309822021_Efficient_Fingerprinting-
Based_Android_Device_Identification_With_Zero-Permission_Identifiers
3. https://www.researchgate.net/publication/
294870168_The_Recognition_of_Fingerprints_on_Mobile_Applications_-
_an_Android_Case_Study
4. https://source.android.com/docs/security/features/authentication/fingerprint-hal
5. https://www.androidauthority.com/how-fingerprint-scanners-work-670934/