, Facial Recognition and Thermal Imaging: A Cost-Effective Solution for …

.


INTRODUCTION
The COVID-19 pandemic, which began four years ago, has presented significant challenges globally.The advent of this global health crisis has necessitated the adoption of a 'new normal' lifestyle, characterized by mask-wearing, social distancing, and constant hand sanitizing.This is an effort to curb the spread of the virus.Mandatory temperature checks have become an essential prerequisite before entry into buildings, shopping centres, and shops, typically conducted using non-contact thermometers.Patients infected with COVID-19 exhibit a range of symptoms, with fever being a common marker.Devices such as infrared thermometers and thermal imaging are employed to detect fever symptoms.The demand for such devices has surged, given their widespread use in temperature checks.Even post-pandemic, these devices are anticipated to remain integral in hotspots like hospitals, clinics, and educational institutions to mitigate the risk of future outbreaks.
In Malaysia, amidst the height of the pandemic, the MySejahtera application [1] emerged as a primary platform for tracking user history.These platforms have the potential to trace hotspot locations and alert if positive cases are detected within a 1 km radius of an individual's residence.This application embodies Internet of Things (IoT) technology as it provides realtime information, inclusive of an additional feature that categorizes the user's location into red, yellow, or green zones.The IoT is a complex network of interconnected computing devices, mechanical and digital machines, objects, animals, or humans each with unique identifiers (UIDs).This network facilitates data transmission without necessitating human-to-human or human-to-computer interaction.This methodology is now prevalently employed, particularly within electronic tracking systems.
In contemporary scholarship, the convergence of thermal monitoring systems and facial recognition technology within the broader scope of building access solutions has elicited significant discourse, predominantly due to its potential to augment security protocols and augment health safety precautions.A subset of these discussions underscores the costeffectiveness of this convergence.The innovative fusion of these two components, whilst considering their financial impact [2] [3], encapsulates two critical dimensions: security validation [4], and health surveillance [5].
Thermal imaging is a specialized technique that detects the long infrared range of the electromagnetic spectrum, invisible to the naked eye [6].This technique employs infrared imagers or specialized cameras.Thermal imaging gained prominence during the outbreak of severe acute respiratory syndrome (SARS) in Southeast Asia, where it was used to screen passengers for fever symptoms [7].For example, the low cost thermal imaging AMG 8833 boasts a detection range of up to 7 meters [8] and the IMX219 CMOS sensor have a large measurable range of −40 to 300 o C [9], while infrared thermometers can measure human temperature at a distance of 0-60 cm from the forehead [5].While infrared thermometers provide a reliable and cost-effective method for non-contact temperature reading, they are limited to single-spot readings [5].Thermal imaging scanners, in contrast, offer faster, more cost-effective, and easier screening.They also provide more accurate subject screening than infrared thermometers.The thermal imaging camera utilizes a focal plane array sensor.
Facial recognition technology, employing unique facial identifiers for personal authentication, offers an expedient and user-friendly mode of access control [2] [10].Concurrently, the incorporation of thermal based monitoring [10,11,12,13] capabilities introduces an auxiliary safety stratum, aiding in the detection of individuals exhibiting elevated body temperatures, a potential harbinger of illness.Scholarly pursuits have concentrated on enhancing the precision and speed of facial recognition algorithms [14] [15].These empirical studies underscore the necessity of factoring in variables such as illuminative conditions, angular variations, and the heterogeneity of facial appearances to ensure reliable and consistent recognition results.Furthermore, the integration of infrared temperature sensors, contactless measurement methodologies, and thermal cameras, has emerged as a viable strategy for precise temperature recording without jeopardizing user experience or safety [11] [16].
The implications of this convergence transcend conventional security precautions, particularly within the context of prevailing global health crises.The capacity to promptly identify individuals with heightened body temperatures could facilitate early detection and containment of contagious diseases, potentially mitigating their proliferation within confined environments.Nevertheless, ethical issues [17][18] pertaining to privacy, data security, and potential biases inherent in facial recognition algorithms necessitate rigorous evaluation in order to achieve equilibrium between security, health, and individual liberties.
The intersection of thermal monitoring and facial recognition within budget-friendly building access solutions suggests a compelling path towards enhancing security and health safety.As technological advancements progress and research in this field evolves, it remains p-ISSN: 2774-602X e-ISSN: 2774-6038 imperative to carefully navigate the technical, ethical, and pragmatic dimensions of this convergence to ensure its effectiveness and broad adoption.
This research is to develop an integrated four-in-one system that can identify and recognize a human face, measure temperature, display real-time temperature readings via the IoT, and maintain a user database for COVID-19 risk assessment analysis.
In the current scenario, even though body temperature measurement before entering a premise is not mandatory, barring identified locations, numerous temperature scanning devices have been developed, boasting advanced features such as facial detection.However, the issue lies in the fact that some of the temperature scanners, though equipped with facial detection, lack facial recognition capabilities.Moreover, traditional temperature scanners, such as infrared thermometers, can only measure temperature and are unable to record the user's temperature data.Furthermore, infrared thermometers can only measure body temperature from the forehead at a very close distance [19].Therefore, a post-COVID-19 tracking database integrated with facial identification and a thermal imaging camera for temperature measurement appears to be a more accurate and practical solution, aligning with the Fourth Industrial Revolution and the IoT.
Moreover, the contemporary era has witnessed the ubiquitous adoption of QR code scanning and temperature measurement in high-risk vicinities, with data being meticulously documented for future reference.As our world continually evolves, so does our technology -an instance of this being temperature scanners equipped with facial recognition capabilities.This system amalgamates non-contact temperature measurement, facilitated by a thermal scanner, with facial recognition technology.Facial recognition is a sophisticated technology that captures an individual's facial features and identifies them.This technology leverages machine learning algorithms to match photographs in a database, enabling the recognition, collection, storage, and analysis of facial features.
The primary objectives of this research are tripartite.Initially, the goal is to design a system capable of recognizing and measuring human temperature within a building, a feature crucial for the early detection of potential health concerns.Subsequently, the research aims to develop a post-COVID-19 tracking database specifically designed for the individuals present within the building.This would expedite the identification and isolation of potential virus carriers, thus ensuring the safety of others within the premises.Lastly, the research will concentrate on assessing the overall performance of the developed system to ascertain its efficacy and dependability.

METHOD
The progression of this project can be delineated into two pivotal phases: the construction of a facial recognition mechanism, followed by the implementation of a human body temperature assessment system.Figure 1 (Left) presents a comprehensive schematic representation of the entire system architecture.The system operates on a Windows platform, utilising open-source Python modules, OpenCV and NumPy, for image recognition processing.The resultant data is then stored in a MySQL server database, managed via the open-source administration tool, phpMyAdmin.The AMG8833 module is deployed in the thermal camera module to gather temperature data from captured human heat signatures.
An ESP32 Wi-Fi module interfaces with the thermal module through 12C interfaces, facilitating the transfer of the ascertained temperature data to the MySQL server.Furthermore, the exchange of data between the thermal imaging sensor and Node-Red employs Message Queuing Telemetry Transport (MQTT), a widely used messaging protocol in the IoT domain.The public HiveMQ broker is utilised to facilitate the transfer of thermal imaging sensor data to the Node-Red dashboard.Node-Red, an integral component of IoT, is a visual programming tool that enables the interconnection of hardware devices, such as thermal image sensors, APIs, and online services.The Node-Red dashboard module allows for the creation of a live data dashboard using Node-Red nodes, primarily for the display of temperature values captured by the thermal imaging sensor.Conversely, Figure 1 (Right) delineates the operational sequence of the entire system, offering a step-by-step illustration of the functional process.

Face Identification
The facial identification system employs intricate computational algorithms to extract specific and distinctive facial characteristics.It leverages open-source libraries such as OpenCV-Python and NumPy, along with Haar Cascade, Python GUI (tkinter), and the MySQL webserver for database management.This system comprises three distinct stages: the data collection phase, the image training phase, and the face recognition phase.Each stage requires a unique programming code and must be executed independently.
The initial stage, data creation, primarily focuses on recording the identification data of an individual.Essentially, the user is required to input their specific details, after which the system prompts the camera to capture 30 sample photographs.These images then undergo a preprocessing phase where they are cropped to isolate the Region of Interest (ROI), which is later employed in the recognition process.
Upon the completion of pre-processing, the images are trained using the Haar Cascade algorithm in preparation for face detection.Once the algorithm is adequately trained, the derived histogram is utilized to represent each image from the training dataset.The Euclidean distance is employed to calculate the distance between the histograms of two separate images.The resulting distance, also referred to as the confidence measurement, is used to determine the similarity between two faces.If the confidence is lower than the established threshold value, it indicates that the algorithm has successfully identified the face.

Temperature Measurement
The methodology employed in assessing the user's temperature involves the utilization of an AMG8833 thermal imaging camera.This camera is strategically focused to capture temperature readings solely from the facial region.The subsequent procedure delineated in Figure 2 provides a comprehensive guide on the development of the temperature measurement system.This system is designed to exhibit the recorded temperature readings on a Node-Red dashboard, providing an easily accessible and understandable representation of the data.

Mathematical Modelling
This segment elucidates upon the theoretical framework and mathematical computations underpinning the process of facial identification and temperature measurement.The rudimentary principles of facial recognition, with regards to a specific algorithm, will be delineated in this segment.An algorithm, referred to as A, employed for recognizing faces, can be construed as a mapping derived from a pair of facial photographs to a real number SA.This number SA embodies a certain measure of similarity (or alternatively, distance) between the two images, as depicted in (1).
where it is common to describe the first image as being drawn from a set of target images ℓ and the second from a set of query images ℓ [5].This definition gives rise to a data structure called the similarity matrix as shown in (2).As delineated by the source [5], the similarity score SA (t, q) is depicted as a match score when the identical individual is depicted in photographs q and t.Conversely, in instances where the respective photographs depict different individuals, the score is referred to as a non-match score.The inference drawn from this observation articulates that the outcome of a verification test is primarily dependent on the similarity score SA (t, q) and the established acceptance threshold.Unlike the verification process, the performance of identification is contingent upon the presence of images and individuals other than the subject within the gallery.
The Haar Cascade, named after Haar-like features, plays a pivotal role in digital image recognition [20].These distinctive features are demonstrated in Figure 3.To facilitate face detection, it is imperative that the Haar Cascade algorithm undergoes a rigorous training regimen aimed at human face detection.This phase can be referred to as feature extraction.The data utilized for the training of the Haar Cascade is housed in an XML file, typically denoted as 'haarcascade_frontalface_default.' In the context of the Local Binary Patterns Histograms (LBPH) algorithm, the Local Binary Patterns (LBP) method and the Histograms of Oriented Gradient (HOG) descriptors are synergistically amalgamated.The LBP method is an efficient, yet uncomplicated technique for both the extraction and labelling of pixels within image structures [21].Specifically, within the realm of face recognition, the Local Binary Pattern Algorithm is deployed.Central to the LBP lies the binary ratio of pixel intensities, located within the central pixel, which typically encompasses eight pixels.Equation 3 delineates the mathematical algorithm employed for the generation of binary code, pertinent to each individual pixel.

𝐿𝐵𝑃(𝑝 𝑐 − 𝑞
In the framework of facial feature determination, the central pixel is denoted as (pc, qc), and the surrounding eight pixels are similarly represented by (pc, qc).This pixel configuration plays a significant role in discerning facial characteristics.Within the face matrix feature, derived from the image in question, there exists a process of comparison between the values of the central pixel and its surrounding counterparts.The outcome of this comparison is a binary code, which is generated as a result of this analytical procedure.

Thermal Imaging
Thermal imaging is a sophisticated methodology employed for the identification of the long-IR region within the electromagnetic spectrum, a region typically imperceptible to the human eye.This detection is facilitated by the utilization of specialized cameras, referred to as infrared imagers.The Long-Wave IR, featuring a wavelength range spanning 8 to 15 meters, generates thermal data or a thermal map of the image captured, presented quantitatively to signify temperature data.Given that the functioning principle of thermal imaging rests solely on the detection of thermal emissions from the image subject, it negates the necessity for ambient light in the acquisition of thermal images.The resultant temperature map is depicted using a pseudo color palette for visual display purposes, achieved through software embedded within the infrared or implemented via a computer system [6].An illustrative example of a pseudo-colored thermal image of a hand can be observed in Figure 4.

RESULTS AND DISCUSSION
The ultimate aesthetics and operational efficacy of each system are assessed through a comprehensive testing procedure.The outcome of the system is bifurcated into three distinct sections for a more nuanced understanding: the creation of the database, the process of face recognition, and the measurement of temperature.Each segment embodies a critical component of the system's overall performance and provides an instrumental yardstick for evaluation.

Database Creation
The collected data, encompassing various forms of information, including the temperature value of the users, has been meticulously documented utilizing the MySQL webserver database.MySQL, a renowned database management system widely recognized for its capacity to house extensive volumes of data, is employed in the present research [22] [23].Figure 5 visually represents the registration data and temperature recording table hosted on MySQL.During the registration process, approximately 23 distinct pieces of information pertaining to both female and male users were amassed.Among these, six individuals identified as IIUM staff, whereas the others were IIUM students.The primary objective behind this data accumulation was to gauge the performance capacity of the MySQL database web server.
The successful recording of all user information validates the efficacy of the chosen method.The table documenting the recorded temperature is intricately linked to the registration table.An interactive feature is incorporated such that by selecting the ID_NO in the recorded temperature column, the database unveils additional information about the user.

Face Recognition
The results derived from facial recognition can be systematically categorized into three distinct segments: data, trainer, and identification.Each segment is articulated through divergent Python programming codes, which are comprehensively laid out in the appendices for further reference.Figure 6 illustrates the creation of a Graphical User Interface (GUI) via Tkinter, ensuring a user-friendly approach to access and operation.Figure 4.A pseudo-colored thermal image of hand [6] The execution of the data segment programming is initiated by the activation of the registration button, with the consequent output represented in Figure 7.
The actuation of the registration button, as depicted in Figure 6, triggers the execution of the corresponding Python registration code.The required user details to be inputted are demonstrated in Figure 7. Immediately after the data entry, the user information is securely stored in the registration table housed within the MySQL web server database, as depicted in Figure 5.The 30 images, subsequently captured, are within a specifically titled dataset folder.These user-specific images are subsequently converted into a grayscale format for further processing.Pertinent user data is compiled along with a collection of 30 exemplar images.An instance of such a user image saved in the dataset folder is displayed in Figure 8.Each image is individually distinguished by the unique identifier, termed as 'User ID_NO'.
Prior to harnessing an image for the task of face detection, two specific equations will be employed to establish a robust foundation.Equation 1, which discerns a particular degree of resemblance in detecting human faces, and ( 2), which constitutes a similarity matrix, will collectively be utilized for the training of human facial patterns.The training phase of the facial recognition system is primarily dedicated to refining the ability of the system to identify specific individuals.This phase involves the creation of a unique histogram for each image involved.The initiation of the training phase is facilitated through the activation of the 'trainer' button, as depicted in the GUI Tkinter Figure 6, which in turn triggers the associated Python programming script.

Figure 5. Registration information database table on MySQL
The effectiveness of the face identification process is illustrated in Figure 9, wherein the system successfully identifies a specific individual.This is achieved through the comparison of the individual's facial characteristics with those captured during the registration process.The individual's ID number, which is stored in the registration database, is subsequently displayed on the screen accompanied by the text "PASS."This process involves the comparison of two histograms, whereby the system identifies the histogram that most closely resembles the dataset and training segments.The algorithm then outputs the ID number associated with the image that has the closest histogram resemblance.Additionally, the algorithm calculates a 'confidence measurement,' which is essentially the calculated distance between the two histograms.The identification process is deemed successful when the confidence measurement falls below the predefined threshold value.
Figure 9 (centre), however, presents the results of facial identification under low-light conditions.Despite the less than ideal lighting, the system still successfully identifies the individual, suggesting that brightness levels do not significantly impact the facial recognition system's performance.All the data used by the system is sourced from the MySQL registration database.The system also demonstrates an impressive tolerance to minor variations in facial expressions.This is exemplified in real-time face recognition scenarios, where the system can accurately identify individuals irrespective of whether they are wearing spectacles at the time of identification, even if their sample photographs were taken without spectacles.The system's algorithm achieves this by analysing the structure of an individual's face and comparing the edges of different facial regions.Figure 9 (Right) presents the results of facial identification when an individual wears spectacle.In addition, Figure 10 represents the outcome of unsuccessful facial identification.The red, square boundary and the text indicating "the record does not exist" appearing on the monitor, signifies that the individual in question has not been registered in the system.Figure 8. Dataset picture for each user Systems that perform real-time face recognition necessitate a processing speed that epitomizes both velocity and efficiency.The developed facial recognition paradigm demonstrates sufficient speed to execute the entire recognition process thrice within a singular second.Optimally, the detection sequence is conducted six times within the same time span.Table 1 delineates the duration required to complete each detection sequence.Upon evaluating the data, it is evident that an average of five detection sequences are consummated within a singular second.The acceleration of the identification process is attributed to the storage of images within the XML (Extensible Markup Language) files in the database.The primary objective of this program is to isolate and analyze an individual's face.Consequently, any supplementary information contained within the images, barring the face, remains unassessed.This selective approach results in a notable reduction in image processing time.Table 1.Time for Recognition

Temperature Measurement
The measurement of an individual's body temperature is effectuated utilizing an AMG8833 thermal camera.As depicted in Figure 11, the comprehensive node-red dashboard display for the thermal camera is presented.With the integration of remote access nodes in the node-red flow as exhibited in Figure 11(Right), the administrator is granted the capability to access and view the node-red dashboard via a mobile device.The node-red dashboard display is segmented into three key sections: the thermal image display, the temperature measurement, and the device status.The highest temperature value in the thermal image 8x8 array is utilized to procure the value.The temperature gauge is situated below, created to log the temperature reading of an individual.
However, the recording of temperature data necessitates manual operation due to a slight disconnection between the facial recognition system and the temperature measurement system.Upon the activation of the submit button, the data is directly saved into the recorded temperature table in the MySQL database.Figure 11 provides an example of the recorded temperature database.
A relationship has been established between this table and the registration table, allowing for the extraction of additional information from the recorded temperature table.On selecting a user's ID number, it directly displays the user's details. Figure 12 illustrates the user's detail as shown when the ID number is selected on the recorded temperature table.The quantitative thermal readings from the AMG8833 thermal camera can be meticulously observed via the serial monitor of the Integrated Development Environment (IDE) Arduino Uno, wherein the temperature is represented in a structured 8x8 array.Additionally, this monitor provides comprehensive reports on the Wi-Fi and MQTT connection status.The thermal measurement system incorporates a pivotal feature designed to alert the administrator through the Remote RED mobile application, as illustrated in Figure 13, should an individual's temperature exceed the threshold of 37.5 o C.This innovative system was initially conceptualized for the purpose of post-COVID-19 risk assessment analysis.One determinant factor for categorizing an individual as high-risk for post-COVID-19 is predicated upon their body temperature reading.The performance of the system is assessed by detecting the thermal variations of a human body from varying distances.The outcomes of these thermal camera examinations are depicted in Table 2.A notable decrease in the temperature reading is observed as the distance between the thermal camera and the human subject expands.The system has the ability to discern human temperatures up to a distance of 7 metres and is proficient in measuring temperatures within the range of 0 o C to 80 o C (32 o F to 176 o F).According to the AMG8833 datasheet [24][25], the accuracy of this temperature detection is ±2.5 o C, equivalent to ±4.5 o F. To evaluate the performance of the AMG8833 system, its temperature readings were juxtaposed with those of the Flir C3 thermal camera.
It is important to mention, however, that this comparative analysis was limited to a distance of 2 metres.The examination was conducted in a room with a constant ambient temperature.The maximum discrepancy observed in the temperature measurements of the two devices was 1.65 o C, while the minimum was 0.7 o C. It was concluded that the AMG8833 system necessitated a value adjustment to align its temperature measurement with that of the Flir C3 thermal camera.This adjustment is due to the AMG8833's temperature accuracy of 2.5 o C, implying that the temperature detected by the AMG8833 could be 2.5 degrees lower or higher.
In critically evaluating the system's performance, it is imperative to consider the economic implications and the accuracy of the system.The total cost of developing the system is approximately RM200, which includes features such as thermal imaging and an image recognition system.In contrast, the Flir C3 solely includes thermal imaging.Despite a discrepancy of 0.7°C at a distance of 0.5m, the system remains practical considering the typical usage of thermal cameras and image recognition systems does not exceed 1 meter.Hence, this system has the potential to be considerably beneficial, particularly in developing countries where both thermal detection and image recognition are required.The system offers nearly 95% cost reduction compared to Flir C3 [26], while maintaining an acceptable level of accuracy and providing additional features.

CONCLUSION
A meticulous comparative examination indicates that the system we designed yields an impressive cost saving of nearly 95% compared to established commercial alternatives like the Flir C3.This cost benefit is further supplemented by the system's image recognition capabilities.The system is robust, amalgamating two primary components: an intricate facial recognition system and an adept temperature measurement system.The former can recognize human faces in a wide array of conditions, while the latter can accurately measure body temperatures from up to two meters away.The latter displays a marginal discrepancy of 0.7°C at a proximity of 0.5 meters, considered feasible given the typical sensing distance is less than a meter.Nevertheless, should the need arise, the system is equipped with the capability to discern human thermal readings at distances extending up to seven meters.The system exhibits proficiency in gauging temperatures within the spectrum of 0°C to 80°C (32°F to 176°F), albeit with a gradual diminution in precision as the distance widens.The facial recognition system exhibits the capacity to identify human faces under a broad spectrum of conditions, while the temperature measurement system can accurately deduce body temperatures from a distance of up to two meters.The system displays the body temperature readings on a specialized node-red dashboard, and a companion tracking database retains relevant personal data for potential analysis of high-risk individuals in a post-Covid-19 scenario.Despite its outstanding performance in achieving the specified goals, the system was hampered by a complex coding error that prevented the simultaneous display of face identification and body temperature.Nonetheless, the overall functionality of the system can be significantly improved by integrating Raspberry Pi and other coding tools.This integration would enable the simultaneous display of face identification and temperature readings on a single screen, thereby enhancing the system's efficiency.Despite the current limitation, the system has demonstrated exceptional performance and met all set objectives.
The training data utilized by the Haar Cascade algorithm is embodied in the XML file -haarcascade_frontalface_default.The initial step in this phase is acquiring the Local Binary Pattern (LBP) of the entire face.These LBPs are subsequently converted into decimal numbers and histograms of these values are then constructed.Ultimately, a single histogram is formed for each image in the training data set.

Figure 1 .
Figure 1.System architecture of the project (Left) Operational Sequence (Right)

Figure 2 .
Figure 2. Process flow to display temperature in Node-Red dashboard

Figure 6 .
Figure 6.GUI Tkinter for face identification system.

Table 2 .
Temperature Measurement Differences Between AMG8833 and Flir C3