Small-area Fingerprint Recognition Based on Improved ORB Algorithm in Embedded Environment

Most of the fingerprint matching algorithms were proposed for large area fingerprints, which can hardly work effectively in small-area fingerprints. In this work, an improved ORB algorithm is proposed for small-area fingerprint matching in embedded mobile devices. In feature descriptor design, we analyzed the characters of the fingerprint in the embedded mobile devices and discard the multi-scale feature process to reduce the amount of operations. Moreover, we proposed a fusion descriptor combing LBP and rBRIEF descriptor. In the key point matching process, we proposed a two-step (coarse and fine) matching method by using Hamming distance and cosine similarity, respectively. The experimental results show that the proposed method has a rejection rate of 6.4%, a false recognition rate of 0.1%, and an average matching time of 58ms. It can effectively improve the performance of small-area fingerprint matching and meet the application requirements of embedded mobile device authentication.


Introduction
As the most common biometric feature, fingerprint is widely used in various identification fields due to its better performance in uniqueness, stability and ease of collection. Its related products hold the number one market share in identity verification identification. Fingerprint identification is divided into registration phase and matching phase. In the registration phase, the fingerprint information of the user is recorded in advance and saved in the template database for using in the matching phase. In the matching phase, the fingerprint image to be matched is compared with the feature template in the database, and if the similarity is higher than the set threshold, the fingerprint can be identified as the same one. Depending on the matching method, the most common fingerprint recognition algorithms are: point pattern based algorithms [1][2] [3] , texture pattern based algorithms [4] and image pattern based algorithms [5] . Among them, the point pattern based fingerprint recognition algorithms are the most widely used.
The traditional fingerprint recognition algorithm based on the point pattern is based on the centroid and singularity for fingerprint image alignment, and then completes the fingerprint authentication by matching the detail feature points, such as endpoints and fork points on the fingerprint. With the increase in portability requirements of mobile devices, the size of their matching fingerprint acquisition sensors has been reduced a lot, and the small-area fingerprint images collected contain fewer detailed feature points [6] , which may lead to missing centroids and singularities, and thus the fingerprint to be matched and the feature template may not be aligned. At the same time, when the overlapping area between the fingerprint to be matched and the feature template is too small, the number of detail feature points available for matching is very small, which makes the traditional fingerprint recognition algorithm based on point patterns have a high rejection rate when dealing with smallarea fingerprint recognition [7] .
Currently, the method of obtaining feature points and their feature descriptors in small-area fingerprint images is generally used to replace the detail feature points in traditional fingerprint recognition methods, and common

Research Article
Jianyu Xiao, Jiuan Liu, Huanhua Liu 2 feature point extraction algorithms include SIFT [8] , SURF [9] , FAST [10] , and Oriented FAST and Rotated BRIEF (ORB) [11] , among others. Among them, ORB algorithm has the advantages of small computation and fast running speed, which is suitable for embedded devices with limited computational resources. However, the correlation of the backward part of its descriptors is high, and it is easy to produce mis-matching when matching similarity by Hamming distance in a single way. Therefore, the original algorithm needs to be improved to ensure the real-time algorithm and to reduce its mis-matching rate. To address the above problems, this paper adopts the following three schemes to optimize the ORB algorithm. Firstly, we will not use image pyramids for multi-scale processing to reduce the computation of the ORB algorithm. Secondly, we introduce rotated LBP descriptors [12] to enrich the feature point neighbor information. Finally, the similarity is calculated by using Hamming distance [13] to select the four points to be matched with high similarity, and then the cosine similarity [14] is used to screen out the points to be matched with large disparity in directional features.
In this paper, feature points and their feature descriptors in small-area fingerprint images are extracted for feature point matching based on the improved ORB algorithm. To overcome the problem of small number of feature matching points due to small fingerprint area, multiple fingerprint images of the same finger are recorded in the registration stage. The relative position information is calculated based on their feature point pairs to generate a fusion template [15] to expand the effective area of the registered fingerprint template, enhance the number of trusted feature points in the template, and reduce the rejection rate of fingerprint recognition.
The rest of this paper is organized as follows. Feature point matching based on the improved ORB algorithm is described in Section II. The small-area fingerprint image registration and matching method based on the improved ORB algorithm is described in Section III. The proposed method is experimented and justified in Section IV, and finally the conclusion is given in Section V.

Feature Point Matching
Feature point matching is the core work of fingerprint recognition. In the registration phase of fingerprint recognition, it is necessary to match the feature points of the fingerprint image to be registered and perform template fusion according to the relative position information of the feature point pairs. In the matching phase, it is necessary to match the feature points in the fingerprint image to be matched and the registration template, and output the fingerprint recognition result according to the number of feature point pairs. The feature point matching process is shown in Figure 1.
The primary job of feature point matching is to select suitable feature points, and then to construct reliable descriptors for matching based on the neighbor information of feature points [16]. Due to the different acquisition directions and finger press positions, the collected fingerprint images may rotate and translation, and the images at different locations may have similar regions, which are prone to mismatching based solely on the Hamming distance, in addition to the weak computational power in the embedded environment becomes a large constraint. Therefore, the selection and matching of fingerprint image feature points and feature descriptors need to satisfy the following conditions: 1) the feature points and descriptors are rotation invariant. 2) They require less computational resources and are suitable for mobile devices. 3) When matching based on descriptors, we need to pay attention to their orientation characteristics in addition to similarity

Feature Point Extraction
ORB is a fast feature point extraction and description algorithm. The ORB feature extraction algorithm is divided into two parts, feature point extraction and feature point description, which uses improved based on FAST feature point detection method (oFAST) as feature point extraction and improved based on BRIEF feature descriptor (rBRIEF) for feature point description [16] . The oFAST is an improvement based on the FAST operator, which determines whether a point is a feature point by detecting the gray value of 16 pixels on a circle with a pixel in the image as the center and a radius of 3 pixels and comparing it with the gray value of that point. As shown in EAI Endorsed Transactions on Internet of Things 02 2022 -04 2022 | Volume 7 | Issue 27 | e5 Figure 2, for a point p in the image, let its gray value be Ip, and if there are N consecutive pixel points on the circle composed of 16 pixels that are larger than Ip+T or smaller than Ip-T, it is considered a corner point, and T is the threshold value, and N is generally taken as 9 or 12. To improve efficiency, the difference between point p and the four positions of 1, 5, 9 and 13 can be calculated first. If three of them are larger than the threshold value, it continues to detect. If three of them are greater than the threshold, it continues to detect other pixel points on the circumference, otherwise discard point p. where I(x,y) is the image grayscale expression, and when p, q take the values 0 or 1, the order 0 moments m00, order 1 moments m10 and m01 are obtained, respectively. The center of mass of this moment is: Assuming that the corner point coordinates are 0, the angle of the vector is the direction of the feature point. The calculation formula is as follows: ORB in practical applications such as face recognition algorithm [17] and fast target detection [18] will achieve scale invariance by extracting feature points under different scale images through image pyramids [19] . After creating an image pyramid, the FAST algorithm is used to find feature points in images of different scales, which eventually forms a collection of feature points on images of different scales. In the fingerprint recognition algorithm, since the fingerprint image is not collected by pressing the finger with much force, most of the acquired small-area fingerprint images have only rotation and translation attributes, therefore, the scale basically does not change greatly. Although the number of feature points obtained is reduced, the missing feature points are all located in the low-scale image, which will not affect the matching of feature points in fingerprint images without scale changes. On the contrary, it can also effectively reduce some of the operations, and the subsequent computation of feature descriptors and performing feature matching are also greatly reduced due to the reduction in the number of feature points.

Feature Descriptors
Feature descriptors [20] are used to describe useful information in an image or image block. According to their preservation methods, they are generally classified into real-valued descriptors (floating-point preservation) and binary descriptors (consisting of 0 and 1). In the image matching process, the binary descriptor is slightly lower than the real descriptor in terms of matching accuracy, but it has the advantages of low time complexity and low memory consumption compared with the real descriptor, which is more suitable for operation in embedded devices with limited resources. rBRIEF descriptor used in the ORB algorithm is a binary descriptor, but since the correlation of the feature descriptors in the rBRIEF descriptor is stronger, it is easy to generate mis-matching. Therefore, this paper uses the LBP feature descriptor to replace the backward part of it to form a fused feature descriptor, which can describe the feature point neighborhood information more accurately.

rBRIEF Descriptors
The rBRIEF descriptor is generated by adding a rotation factor to the BRIEF feature descriptor [21] and reducing its relevance by a greedy search. The idea is to take a certain number of point pairs in the neighbor points detected by the oFAST algorithm and generate feature information based on their gray value magnitude. Its testing criterion is: In Eq. (4), p(x), p(y) are the point gray values of the key point neighbor, x and y are the point pairs selected according to certain laws, n such point pairs form an n-dimensional binary vector.
Since the BRIEF generated at this time does not have rotation invariance, of which the matching performance decreases sharply when the rotation angle is greater than 30°. The original ORB algorithm rotates all the sampled points in the main direction of the feature points extracted by oFAST, so that they have rotation invariance. First, n pairs of points (xi,yi) are selected in a certain neighbor center on the feature points in a certain way, forming a 2×n matrix as in Eq.
The specific operation is shown in Figure 3. The coordinate axes are rotated by θ, and the matched point pairs with the main direction θ as the coordinate system are calculated, thus generating the streered BRIEF descriptors with rotational invariance. In the original ORB algorithm, the correlation between the streered BRIEF descriptors generated after the rotation of the point pairs selected using Gaussian distribution will be greatly increased, which is not conducive to feature matching. Therefore, a greedy search is proposed to replace the Gaussian distribution to select point pairs to reduce the correlation. The principle is to search all point pairs with larger variance and mean value close to 0.5 as the sampled point pairs. To form a sequence of n columns of global optimal point pairs, generate an n-dimensional binary vector, which is the rBRIEF descriptor, and n can generally be taken as 128, 256 or 512, and its form is shown in Eq. (8).

LBP feature descriptors
Local Binary Pattern (LBP) feature is a powerful method to describe texture features, which can measure and extract texture information from local neighbor regions in an image, which has been widely used in face recognition. The main principle is to form a binary descriptor by comparing each gray value of a local image window with the central pixel gray value. The LBP descriptor operator is defined as: In Eq. (9), P and R are the number of surrounding sampled pixels and the sampling radius, respectively. ti is the gray value of the surrounding pixels, tc is the gray value of the central pixel, s(x) is a function of the gray value which is to quantify the relationship between the surrounding pixels and the central pixel. A sample calculation is shown in Figure  4. The LBP feature descriptors generated in this way do not have rotational properties, so the LBP operator can be shifted and its minimum value can be taken as the final LBP descriptor with rotational properties. Here we take P as 16 and R as 2, then the final 16-bit LBP feature descriptor can be generated.

Fusion Descriptors
Since rBRIEF is taken according to its correlation size, the more forward descriptors have less correlation and the more backward descriptors have more correlation, which do not sufficiently utilize the neighbor information of feature points and lead to mis-matching when matching feature points. rBRIEF-LBP feature descriptors can effectively describe the neighbor information of feature points and have rotational invariance after shifting them to obtain the minimum value. Therefore, in this paper, the rBRIEF-LBP fusion descriptor [22] is formed by generating a 16-bit rotational LBP descriptor to replace the back 16 bits of the rBRIEF descriptor without changing its rotation-invariant property, which can enrich the neighbor information of feature points and reduce the feature point mis-matching rate with a very small operation cost. Figure 5 shows the fusion descriptor generation process.

Feature Point Matching
In the original ORB algorithm, the Hamming distance of the rBRIEF descriptor is directly calculated to determine whether it is the corresponding feature point. The Hamming distance is much more efficient than the similarity algorithms of Euclidean distance, Marxian distance [23] and information entropy [24] in terms of computational efficiency, but it also has a major drawback: the Hamming distance can only be used to measure the similarity between two pairs of feature points, and when there are more similar regions in the image, it is easy to produce a mis-matching [25] . In this paper, we use a combination of both Hamming distance and cosine similarity, using Hamming distance to reflect the difference of vectors in numerical features and cosine similarity to reflect the difference of vectors in directional features.

Hamming Distance
The Hamming distance represents the number of bits that are different between two strings, and we denote the Hamming distance between two strings x, y by d(x, y). The experimental data show [26] that when the number of candidate matching points is 4, it can ensure the average recognition time and improve the correct matching rate, i.e., when the number of candidate matching points is greater than 4, the top 4 arranged are selected as the candidate matching points.

Cosine similarity
The cosine similarity is calculated by calculating the cosine value between two vectors to measure the magnitude of their differences in spatial directional characteristics. The cosine similarity of two vectors in the 3D coordinate system is shown in Figure 6.

Feature Point Matching Program
By calculating the Hamming distance of the rBRIEF descriptor, the feature points with closer similarity are screened as the candidate feature points, and then the optimal feature point pair is derived by calculating the cosine similarity. The matching process of the specific feature point pair is as follows.
Step 1: Select a feature point from the set of feature points of the fingerprint image P as the feature point to be matched.
Step 2: Calculate the Hamming distance between the 240bits rBRIEF descriptors of the feature point to be matched and the rBRIEF descriptors of all feature points of the fingerprint image Q. The top 4 feature points in terms of similarity are selected as the candidate feature points [27] .
Step 3: Calculate the cosine similarity of the feature points to be matched and the LBP feature descriptors of the candidate feature points, respectively. Set the threshold value, and if the cosine similarity appears to satisfy the set threshold value means that the feature point pair is matched successfully, if there are more than one feature point satisfying the condition, the feature point pair is formed by merit, and vice versa the feature point pair matching fails.

Small-area fingerprint recognition
Small-area fingerprint recognition [28] is divided into two processes, fingerprint registration and fingerprint matching, just like traditional fingerprint recognition, and the process is shown in Figure 7.  Template fusion [29] .
In the fingerprint recognition stage, after obtaining the feature points and feature descriptors of the fingerprint image to be recognized, the feature points are compared with the feature points in the fusion template for coarse and fine comparison. When the number of feature pairs meets the given value, the fingerprint match is considered successful, otherwise the matching fails. area between images is too large, the template fusion will result in too many overlapping feature points and cannot achieve the purpose of increasing feature points. If the overlap area between images is too small or no overlap at all, the number of feature point pairs is too small for template fusion. Therefore, after recording five fingerprint images, the similarity between images needs to be calculated separately and the threshold values T and F are set. If the similarity between the existing template and all other templates is less than T, the relative position of the image cannot be determined and a fingerprint needs to be re-entered to replace the template. Otherwise, the two images are the same location and also need to be re-entered.

Table 1. Example of template similarity
The benchmark template is selected in two ways: 1) The template with the highest average similarity is used as the benchmark template [30] . 2) The overall optimal strategy [31] .
We use the overall optimal strategy to select the benchmark template. In addition, each splicing template is selected from an overall perspective. The template with the highest total number of alignable templates among the five templates is selected as the base template. If the similarity threshold is set to 0.2, although the average similarity of template F3 is lower than that of F2, its total number of alignable templates is 4, which is higher than that of F2. Template F3 has reference significance for the alignment of the rest of the templates.
Obviously, it is more appropriate to select F3 as the benchmark template.

Template Fusion
The user only needs to press gently to get a clear fingerprint

Experimental environment and data set
The experiments in this paper use MediaTek's MT2503 processor with 32M bit of memory and Nucleus as the

Feature point matching effects
In order to verify the variation of the number of feature points extracted by the improved ORB algorithm in small-area fingerprint images and its rotation invariant property, the small-area fingerprint images in the database are taken for feature point extraction using two methods and rotated to match the feature points with the original image, and the results are shown in Fig.10. Fig.10 (a) is the original fingerprint image, Fig.10 (b) is formed by rotating Fig.10 (a) by 180°. In Fig.10 (c), the scale operation is performed on the original fingerprint image using the image pyramid, and the total number of feature points extracted on the 4-layer pyramid is 476. In Fig.10 (d), the scale operation is not performed on the fingerprint image, and the number of feature points extracted directly on the original image is 308.
The number of feature points is greatly reduced, and the subsequent computation of feature point matching is also reduced. Fig.10 (e) shows the effect of feature point matching for (a) and its rotation by 180° to generate figure   Fig.10 (b), which shows that the improved ORB algorithm still has better rotation invariance.

Accuracy Verification
In order to verify the accuracy of the proposed algorithms, the following four algorithms were tested on the self-built small-area fingerprint database, respectively. shows that the template stitching method can improve the effective area of small-area fingerprint images and the number of feature points, thus reducing the rejection rate. 2) Algorithm Trad and algorithm ORB-N-SPL take 158 ms and 93 ms to match a single fingerprint respectively, which is acceptable. Because the first two algorithms do not perform fingerprint template stitching and need to compare the recorded fingerprint images one by one during matching, which will further increase with the increase of the