Abstract

1. Introduction

In this paper, we describe the development of a security system that leverages FPGA and IOT technologies with face recognition capabilities. Specifically, we utilize the openCL library and the HLS tool for converting C++ code into VHDL so that it can be deployed on FPGA.

Our system is designed to authenticate individuals and grant access to a building or office using facial recognition technology. If an individual attempts to bypass the authentication process, the system can detect this in real-time and alert security personnel. This feature is particularly useful in high-security environments where unauthorized access can have serious consequences.

In addition to authentication and access control, our system also includes real-time face detection capabilities using all the cameras integrated with our IOT system. The system compares the detected faces with those stored in a database and alerts security personnel if there is a match. Our system can also locate a specific person in the building, provided they are within the camera's view. This feature is helpful for surveillance and finding someone.

Overall, our security system has the potential to improve security and reduce the risk of unauthorized access to buildings and offices. It is a powerful tool that can be used in a variety of settings, from high-security government buildings to commercial buildings and residential areas. The use of FPGA technology provides a fast and efficient processing system that can handle large amounts of data, making our system reliable and effective.

Facial recognition is a means of authenticating or validating a person's identity through their face. Software for facial recognition can recognise people in real time as well as in still photographs and movies. Face recognition is a subtype of identity verification. It works by detecting and measuring facial features in photos. Techniques for facial recognition may identify human faces in photographs or videos, determine whether a face appearing in two distinct shots taken by the same individual, or search for a face in a large database of previously captured pictures. Monitoring and security applications are among the most popular application of real time monitoring and face recognition technologies because boost productivity and dependability while reducing human error.

We can improve the accuracy and reduce the latency of our surveillance system by using FPGA. As we can see that there will be a lot of camera’s in our system so they all need to work together which require creating a number of different instances for each camera on our processor for further face recognition and then generating output which will be the alert system or as required by the condition so to overcome that problem we will be taking help of parallel processing power of FPGA. Which means that we can create as many instances as we want for our system so that they can work simultaneously to give better result.

The Iot on Ai - powered system connecting computer vision to the IoT ends up creating strong security; it helps by providing real-world environments with speedier, better support with much less efficiency time, as well as all systems are linked to the internet with necessary technological systems that support more effective and precise operations. Scalable frameworks are needed for IoT systems. Iot systems can readily communicate with the external world with low power, reduced latency, and excellent determinism by the usage of Field Programmable Gate Array (FPGA).

2. Background   

3. Project Work

Development and implementation of an IoT-based real-time security monitoring system: An affordable IoT-based monitoring and security system that has applications for home security and other forms of management is presented. The system employs Rpi and Wifi cameras for the detection, reporting, and user surveillance of incursion incidences. The system alerts users and notifies the region, reducing the damage caused by burglaries. The system also utilizes cloud architecture for the storage of acquired photographs and videos.

Real-time surveillance using IoT and computer vision: The suggested system employs computer vision to detect people in real-time surveillance using IoT. With the appropriate geolocation, the system alerts the police. The system performs real-time face identification by detecting faces in live human face streams and labelling them with the name of the verified individual. If an individual is not identified inside the database, real-time face matching labels the person as "UNKNOWN." Real-time face recognition systems are used in video surveillance to track a single individual in real-time.

Remote DeepFace Models Face Recognizer Creation and Implementation Using the sbRIO FPGA Technology and the NB-IOT Module: A remote DeepFace model system for face recognition using the camera module is created. The system integrates sbRIO FPGA with NB-IOT module. The test findings are quite promising. Based on the NB-IOT module, a distant network infrastructure and local application server are created, and their suitability for data transfer is tested. The DeepFace model's FPGA implementation is completed, which has application potential.

OpenCL based FPGA Accelerator Development and implementation for YOLOv2: The development and implementation of a YOLOv2 FPGA accelerator built on OpenCL are presented. The convolutional layer is processed using the Winograd technique to improve the model's detection speed. The convolution procedure is sped up by highly parallel processing, significantly improving efficacy compared to prior relevant investigations.

Our system will have two features. One feature will be alerting if an unauthorized person enters the premises, and the other feature will be finding a person in the building or tracking a particular person in the building or an area wherever our system is implemented. Firstly, all the cameras will be giving us live video, which will then be fed to our FPGA using Wi-Fi modules. The FPGA is specifically programmed for the task of face recognition, which will then transfer its result as an output for our IoT module, which will be Arduino or Rpi in our case. Then RPi will generate the alert message required for the given condition. For example, if any unauthorized person was spotted in the building, then it will send a message containing a warning and Facial ID with the location of that person so that security persons can reach there and identify the intruder quickly. If we have selected to find a person and when the person is found, then it will send a message containing the location of the spotted person to the concerned authority.

The entire procedure is implemented as IoT using a linked FPGA, Raspberry Pi 3, and camera. On Vivado HLS, we natively synthesized a C++ model of our code to produce HDL files for FPGA implementation. Our system was created utilizing the LBPH OpenCV recognizer. The human mind finds it extremely easy to recognize faces, but in the computer vision pipeline, we must first collect the data, then analyze it, and last train and educate the model to recognize different facial features.

Since Eigenfaces & Fisherfaces, these two recognizers are also influenced by light. We chose LBPH since we cannot always guarantee ideal lighting in real life. The LBPH face recognizer is a development that addresses this flaw. The idea is to focus on an image's local elements rather than its overall composition. The LBPH method compares each pixel with its surrounding pixels in order to determine the local structure of a picture. After constructing a list of LBP, we convert each binary image to a decimal value before generating a histogram that includes all of the values.

The training data collection will ultimately contain one histogram for each face image. They are then kept for further identification. For the reason that the algorithm maintains track of which histogram corresponds to which individual. When we input a new picture to a recognizer for identification, the recognition system will produce a histogram with that new image, compare it to the histograms that it already has, select the best match histogram, and provide the individual label associated with that best fit histogram.

4. Result

5. Conclusion

Real-time surveillance video is sent into an FPGA for face extraction and recognition, and the result is fed into an IOT module, which hosts an application that sends alarm messages to security and other concerned parties through their mobile phones and desktop systems. As illustrated below, our facial recognition technology was able to discriminate between approved and unauthorised users. When an unauthorised individual was detected, our system displayed a red box around the intruder face, as well as an intruder alert message on the desktop screen. After detecting an illegal person, the system transmitted an alarm signal via a web application, as illustrated, with the intruders location derived from the position of the camera on which the person was detected. The real-time face recognizer is used in security surveillance to monitor a specific individual in real time. This technology is beneficial when security staff are unable to watch 24 hours of Surveillance tapes. AI-like capabilities are possible with computer vision. AI on FPGA thinks smarter and quicker than previous systems where AI processing is done just on a Raspberry Pi or a standard CPU. From the result we can conclude that the integration of raspberry pi and FPGA makes our system better than ever before where we either used raspberry pi or CPU with high processor for computation and processing of face recognition algorithm and for communicating the result of algorithm throughout the network.

In this research, we provide a real-time face detection and recognition surveillance system that executes face extraction and identification on FPGA utilising HLS synthesis. This system was created to detect an intruder in the building or any other place where the presence of an unauthorised person poses a threat. The dataset on our system contains all of the information about authorised people, including images that are used to recognise their faces. If an unauthorised person is spotted on any camera, our system generates an alert message and sends it to all of the people who are concerned about the issue, thanks to the IOT infrastructure that we have implemented on our system with Raspberry Pi. Our technology, which is used to host an application through which alarm messages are transmitted, may also be used to discover a certain individual and when the system does notice that person on any camera. Preliminary findings of the built system show that it is capable of doing all of the tasks described in a highly fluent way. Nonetheless, we have created a noble system with increased accuracy and reduced latency for greater usefulness and reliability.

6. Acknowledgement

7. References

I would like to express my gratitude to my teachers, Prashant Pal, Shashank Kumar, Pawan Kumar Patel, and Saurabh Bhansod, for their guidance and support throughout the completion of my project and research paper titled "Realization of Smart and Highly Efficient IoT based Surveillance System using Facial Recognition on FPGA". Their extensive knowledge and unwavering encouragement have been crucial in shaping my understanding of the topic.

I also want to extend my appreciation to my colleagues and peers who provided constructive feedback and suggestions, contributing to the overall quality of the project. Without their support and inputs, the research paper would not have been as successful.

Additionally, I want to thank the institutions and organizations that provided resources and infrastructure essential for the research. Their contributions have been instrumental in the completion of the project.

In conclusion, I want to express my gratitude to everyone who played a part in the successful completion of this project and the publication of the research paper. Their unwavering support and encouragement have been vital in achieving this milestone.

1. Wen, L., "FPGA-Based Deep Convolutional Neural Network Optimization Method," 2021 International Conference on Signal Processing and Machine Learning (CONF-SPML), 2021.

2. Yang, A. et al., "An OpenCL-Based FPGA Accelerator for Compressed YOLOv2," 2019 International Conference on Field-Programmable Technology (ICFPT), 2019.

3. Babu, P. and Parthasarathy, E., "Optimized Object Detection Method for FPGA Implementation," 2021 Sixth International Conference on Wireless Communications, Signal Processing and Networking (WiSPNET), 2021.

4. Cui, C. et al., "Design and Implementation of OpenCL-Based FPGA Accelerator for YOLOv2," 2021 IEEE 21st International Conference on Communication Technology (ICCT), 2021.

5. Fujiki, D. et al., "SeedEx: A Genome Sequencing Accelerator for Optimal Alignments in Subminimal Space," 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), 2020.

8. Developed by