Remote consultation image stitching method based on wireless sensor technology and mathematical morphology on Pervasive and Technology

INTRODUCTION: In order to obtain seamless and high-precision remote consultation image mosaic results, a remote consultation image mosaic method based on wireless sensor technology and mathematical morphology is studied. A consultation image acquisition unit based on wireless sensor technology is designed, and the remote consultation image signal is collected by sensor; In the process of signal conditioning, a filter based on mathematical morphology is used to reduce the influence of noise on the accuracy of remote consultation image acquisition. OBJECTIVES: Compressed sensing technology is used to realize the compression, transmission and recovery of consultation image sampling data. METHODS: After preprocessing the image through shadow correction, surf algorithm is used to construct the scale space to determine the main direction of feature points in the image; The extended surf descriptor is constructed based on feature points for consultation image registration. RESULTS: Based on the spatial transformation relationship between images, the improved gradual in and gradual out stitching method is used to complete the remote consultation image stitching. Experimental results show that this method can accurately collect consultation image signals, and the corresponding rate of the feature point extraction results reaches nearly 99%, which is relatively robust. CONCLUSION: The RMSE error of the image registration results is less than 2.692, which improves the accuracy of the remote consultation image stitching results, well solves the problem of image visual field reduction, and there is no seam in the stitching area.


Introduction
Telemedicine consultation technology is to connect the computers of the two hospitals with communication lines to realize remote transmission of patient data, CD films, X-rays, MRI films, slides, B-ultrasound, ECG, etc. in different places, and implement the consultation means of Visual Dialogue [1]. Seamless image stitching technology is a hot spot in the field of computer image processing in recent years. Image stitching is to combine several partially overlapping images into a seamless and smooth transition, high-quality panoramic image [2], and its stitching quality greatly depends on the degree of image registration [3]. How to automatically and quickly complete the stitching of high-precision remote consultation panoramic images in the process of remote consultation is one of the research hotspots in the field of image processing. A basic requirement of high-precision remote consultation panoramic image stitching is that the structure and color of the stitching between images can be EAI Endorsed Transactions on Pervasive Health and Technology 03 2022 -07 2022 | Volume 8 | Issue 31 | e5

Research Article
Li Xiaoge 2 seamlessly and naturally transferred [4]. However, in general, the stitching gap between images is inevitable without special processing. The causes of stitching gap can be divided into two categories [5]. One is the brightness change caused by inconsistent exposure; The other is the geometric structure gap caused by the error of image registration, such as separation, dislocation or overlap.
Shi et al. studied the image stitching method based on the semantic feature extraction of convolution neural network [6], and calculated and quantified the semantic features of each pixel in the image through neural network to represent the contribution of each pixel to image semantics. According to the quantization results, the semantic contribution value of each pixel was sorted, and the semantic feature points were selected from high to low to complete the image stitching. However, the calculation speed of this method is relatively slow, and it can not deal with large-scale image stitching ideally. Tian et al. studied a new method of spaceborne scanning SAR image stitching based on homography matrix compensation [7], and preprocessed the image through an improved Wallis filter to eliminate intensity inconsistency. The overlapping region of adjacent images was roughly extracted by geographic location technology. Through homography matrix compensation, downsampled image registration and original size image projection were realized, so as to complete image stitching. However, this method is only suitable for scenes with low accuracy requirements. Chaitra et al. studied the image stitching by using feature detection and matching techniques [8].
In order to improve the accuracy of the results and obtain seam-free stitching images, this paper studies the remote consultation image stitching method based on wireless sensor technology and mathematical morphology. In view of the difficult data acquisition or the high demand for stitching accuracy, which requires that the image stitching error can be corrected according to the existing data, and the high-precision stitching results of remote consultation image can be obtained by the method proposed in this paper. Innovatively reduce the influence of noise on the image acquisition accuracy of remote consultation through the filter based on mathematical morphology; The compression, transmission and recovery of consultation image sampling data are realized by compressed sensing technology. The surf algorithm is used to construct the scale space and determine the main direction of feature points in the image; Build extended surf descriptor based on feature points to complete consultation image registration; Based on the spatial transformation relationship between images, an improved gradual in and gradual out stitching method is adopted to realize remote consultation image stitching.

Materials and methods
The remote consultation image stitching method based on wireless sensor technology and mathematical morphology is mainly realized by image information acquisition and processing at the remote consultation image acquisition end and image processing and stitching at the remote computer end.

Consultation image acquisition
Design of consultation image acquisition unit based on wireless sensor technology Figure 1 shows the structure of consultation image acquisition unit based on wireless sensor technology, which mainly includes main board, RF communication module and sensor module.
In the consultation image acquisition unit based on wireless sensor technology, the main board performs analog-to-digital conversion and necessary signal processing on the analog acoustic signal and optical signal from the sensor module [9], and completes the functions of consultation image data compression, network communication, power management and so on. In order to support the storage and analysis of large data streams generated by acoustic and optical signal acquisition, especially the requirements of data compression operation on calculation and storage resources, the main board hardware core selects the 32-bit digital signal processor TMS320F2812 with the main frequency of 150 MHz, and uses its rich chip resources and interfaces to expand 2M byte SRAM, 32K byte EEPROM, 8-channel audio interface, ZigBee communication module interface, RS232 serial interface, etc, to increase the flexibility and expansion of the motherboard.
The RF communication module is mainly responsible for sending and receiving consultation image data packets and network management [10]. In order to transmit the sampling consultation image data with large flow in real time and reliably, and reduce the difficulty of software and hardware design, the RF communication module is selected to comply with IEEE802 15.4 standard, lowpower RF communication module MRF24J40MA supporting ZigBee protocol.
The sensor module mainly realizes the functions of acoustic signal and optical signal conversion, pre amplification, filtering and so on [11]. The sensor module is composed of sound sensor probe, optical sensor probe and signal conditioning circuit. The module can directly use the sound sensor (microphone, etc.) and optical sensor in the unit shown in Figure 1. In order to improve the detection performance, the consultation image acquisition unit based on wireless sensing technology uses the sound sensor of PZT piezoelectric ceramic ring, laser sensor and self-made signal conditioning circuit to form the sensor module.  The main technical parameters of the acquisition node are as follows: the diameter of the sound sensor probe is 30 mm; Length is 100 mm; Frequency response range is 10 ~ 30000 KHz; Input impedance is >2000 MΩ; Output impedance is ≤50 Ω; Voltage gain is 60 ~ 100 dB; The sound sensor probe and signal conditioning circuit are connected by low-noise coaxial cable, and the distance is ≤ 60 m. The DC power supply of the optical sensor is 10 ~ 30 VDC; The maximum power consumption is 0.4 W; The range of light intensity is 0 ~ 65535 Lux, 0-200000 Lux; The temperature and humidity range is -40 ℃ ~ 60 ℃, 0% RH ~ 80% RH; The output signal is RS485 (Modbus Protocol). The transmission rate of RF communication module is 250 Kbit/s, the working frequency band is 2.40 ~ 2.48 GHz, the typical sensitivity is -102 dBm, the maximum RF input is -23 dBm, the typical output power is 20 dBm, the control range of transmission power is 56 dB, and the transmission distance is ≤ 1000 m. The allowable sampling frequency is ≤6 MHz, the sampling accuracy is 12 bits, and the power is supplied by 12~36 VDC power supply.

Filter design based on mathematical morphology
In the signal conditioning process of consultation image acquisition unit based on wireless sensor technology, in order to reduce the impact of noise on the accuracy of remote consultation image acquisition, threshold method can be used to improve the accuracy of remote consultation image acquisition [12]. However, this method also has some defects. Its main disadvantage is that the preservation of feature information of remote consultation image depends on the assignment of gray segmentation threshold . When the selection of gray segmentation threshold is not reasonable, not only the noise points will be filtered out, but also some other information related to the consultation image will be eliminated, which will affect the accuracy of consultation image acquisition. To solve this problem, based on the threshold selection method, a filter based on mathematical morphology is designed and applied to the signal conditioning circuit of consultation image acquisition unit based on wireless sensor technology.
(i) Threshold selection, The target of threshold selection: the gray value of all noise is less than an optimal threshold , so as to remove all noise in all consultation image signals. From the abovementioned methods and analysis, it can be concluded that the selection of threshold should not be too large to minimize the loss of useful information related to consultation images. Nevertheless, the threshold determined in the general binarization method does not reach the ideal value. (a) The remote consultation image signal is binarized with the initial threshold 0 0 U = , and then the morphological filter is used to eliminate the noise of the remote consultation image signal [13]. (b) If the noise is not eliminated in this step, the remote consultation image signal is binarized by 10 1 UU =+ again, and the morphological filter is applied again. (c) It should check whether the noise is completely eliminated. If not, continue with step (b). The detection standard is that when the two denoising effects of UN and 1 N U + are basically the same, the noise can be considered to be completely eliminated. This step will be performed 1 N + times until there is no noise in the binarized remote consultation image signal. (ii) Morphological filter design, the basic idea of mathematical morphology is to collect image information using a probe called structural element [14]. When the probe moves continuously in the image, the structural element is used to detect the image to determine whether the structural element can be filled into the interior of the image.
At the same time: Where,  represents a morphological operation, and the  represents the result of deriving the right from the left.
Formulas (2) and (3) make the structural element i R must maximize the elimination of the noise of the original remote consultation image and minimize the distortion of the image signal [15]. According to this feature, the general structure of the filter is as follows: Where, and  represent the opening operation and closing operation in morphological operation respectively.

Compression and recovery in the process of consultation image data transmission
The bandwidth of wireless sensor network based on ZigBee is limited. The acquisition node needs to compress the sampling data of remote consultation images with large flow to ensure real-time transmission. In addition, this method of increasing the amount of calculation in exchange for reducing the amount of communication can effectively reduce the energy consumption of the signal acquisition node of remote consultation image and prolong the life cycle of the node. Therefore, the core content of the software design of remote consultation image acquisition unit based on wireless sensor technology is the design of data compression and recovery algorithm. The existing data compression algorithm has complex structure and large amount of calculation, which is difficult to be directly transplanted to the acquisition node with limited resources. Therefore, the compressed sensing technology is used to realize the compression, transmission and recovery of remote consultation image sampling data.
In the signal conditioning process of consultation image acquisition unit based on wireless sensor technology, in order to reduce the impact of noise on the accuracy of remote consultation image acquisition, threshold method can be used to improve the accuracy of remote consultation image acquisition [12]. However, this method also has some defects. Its main disadvantage is that the preservation of feature information of remote consultation image depends on the assignment of gray segmentation threshold . When the selection of gray segmentation threshold is not reasonable, not only the noise points will be filtered out, but also some other information related to the consultation image will be eliminated, which will affect the accuracy of consultation image acquisition. To solve this problem, based on the threshold selection method, a filter based on mathematical morphology is designed and applied to the signal conditioning circuit of consultation image acquisition unit based on wireless sensor technology.
Considering the finite one-dimensional discrete-time signal x in the real number field, if there is a set of sparse bases  , so that formula (5) holds, x is said to be K sparse on the base  .
= xa   In formula (5) By solving the optimization problem in the sense of 0 l norm shown in formula (7), the exact or approximate approximation x of x can be obtained, that is: In formula (7): The dimension M of the measured value is much smaller than the signal dimension N , and the optimization problem (7) is an underdetermined problem [16]; However, by reasonably selecting the measurement matrix  and sparse basis  and making them meet the finite equidistant characteristic criterion or irrelevant, formula (7) (8) is a convex optimization problem, it can be easily transformed into a linear programming problem, which can be solved by interior point method, gradient projection method, second-order cone programming, matching pursuit method and so on. The basic property of linear programming is to maximize or minimize the objective function under some constraints. The objective function is a linear function obtained from the mathematical model of the problem. Convex optimization is a kind of special optimization, which refers to a kind of optimization problem in which the objective function of finding the minimum value is convex function. Therefore, the convex optimization problem can be transformed into a linear programming problem.
In the design of acquisition unit of remote consultation image based on wireless sensor technology, sparse basis  selects fast Fourier transform, measurement matrix  selects Gaussian random matrix, and the signal recovery algorithm of remote consultation image selects interior point method; Signal dimension N and measured value dimension M are determined according to the test; (5) The calculation of formula (6) is carried out on the acquisition node of the consultation image acquisition unit based on wireless sensor technology, and the calculation of formula (4) and (5) is completed on the remote computer.

Remote consultation image preprocessing
After receiving the remote consultation image signal, the remote computer generally needs to carry out certain preprocessing before stitching to obtain better stitching results. It is worth mentioning that in the process of remote consultation image acquisition and a series of preprocessing of remote consultation images, the consistency of various parameters and conditions must be maintained, or necessary optical density and scale correction must be carried out to remove the inconsistency brought by the process of acquisition and processing.
In the process of remote consultation image acquisition, due to the uneven illumination in the field of view, that is, the background of the remote consultation image has uneven gray value, which will cause a certain difference between the gray value of pixels in the remote consultation image and the actual value. In the remote consultation image stitching, the same target is displayed as different gray values in the overlapping area of adjacent images, which brings great difficulties to the accurate matching of remote consultation images. Moreover, in the final stitched image, the gray difference at the seam is very obvious. Shadow correction is to correct the non-uniformity of this background.
The background image without specimen collected during remote consultation image acquisition is

Image registration
After preprocessing the remote consultation image, the SURF algorithm is used to construct the scale space to determine the main direction of the feature points in the image. Surf adopts the approximate value image of Hessian matrix determinant. There are some overlapping layers in the adjacent groups of scale space of surf algorithm. At the same time, the size of box filter in each group increases gradually, which maintains the continuity of scale space and is conducive to the subsequent seamless splicing operation. Constructing extended SURF descriptor based on feature points, and on this basis, feature matching is implemented to realize remote consultation image registration.
In the process of constructing scale space by SURF method, the size of remote consultation image does not change, but the filter changes. The calculation accuracy and efficiency are improved based on block filtering. The small to large block filter is convoluted with the original remote consultation image to form an image pyramid, that is, scale space.
The feature points with fixed scale and rotation are obtained on the corrected remote consultation image through exclusion, comparison and interpolation. The specific process is as follows: excluding the points less than the threshold and retaining the points with the strongest response to reduce the amount of calculation; Comparing each pixel and its adjacent points in the scale space to determine the extreme point, that is, the candidate feature point; The sub-pixel accuracy of feature points is determined by interpolation in space and scale space.
The gray distribution of pixels near different feature points detected in the same scale is represented by SURF descriptor [17]. The integral image is convoluted with the filter to obtain the gradient values in x direction and y direction, so as to improve the robustness and operation Feature matching is implemented based on feature points and extended surf feature descriptor. When determining the similarity between the reference image and the key points in the image to be matched, the Euclidean distance of the feature vector of the key point descriptor is selected as the standard [19], and the matching point pair is more stable by reducing the proportional threshold of the two key points closest to the Euclidean distance.
The homogeneous coordinate projection transformation matrix between the reference image and the image to be matched is as follows: and vertical displacement respectively. RANSAC algorithm and least square method are used to remove the wrong feature point pairs and obtain the spatial transformation relationship between the registered images.

Image stitching
The transformation matrix obtained in the previous section is used to realize image stitching and fusion. It needs to select a reference image and a composite surface. The reference image is selected as the center image of the scanned image, which can reduce the stitching error. A flat synthetic surface is selected because the remote consultation image ultimately has only transformations that belong to projective transformation. The image to be matched is converted to the reference image coordinate system by using the transformation matrix.
Once all remote consultation images are mapped to the composite surface, appropriate methods must be selected to splice them into a high-quality panoramic remote consultation image. Image stitching includes average method, linear transition of re pasting region, multiresolution spline technology and so on. If all images are fully registered and have the same exposure, the average method can be simply used. However, in practice, there must be a certain degree of poor registration and uneven exposure [20]. As a result, the panoramic remote consultation image may have visible seams or blur. Direct using the traditional method of gradually in and out of image fusion will produce obvious splicing traces, cause fusion area transition is unnatural, so from the algorithm of the complexity and fusion image quality, using a improvement of splicing method, control the overlap pixel gray difference, improve the overlap area image splicing smoothing effect. The algorithm is shown in formula (14):

Results
In order to verify the application effect of the remote consultation image stitching method based on wireless sensor technology and mathematical morphology in the actual remote consultation image stitching, the proposed method is applied to an affiliated hospital to conduct remote consultation for a patient with fundus vascular lesions. This method is used to test the stitching of the patient's remote consultation image. The method based on convolutional neural network in reference [6] and single stress matrix compensation in reference [7]. The results are as follows. According to the analysis of Figure 2, the signal acquisition results of the method in this paper are basically consistent with the actual noise-free signal, and the reference [6] method of acquisition results are quite different from the actual noise-free signal, which shows that this method can effectively suppress the noise interference in the process of image acquisition, accurately collect the consultation image signal of patient's fundus vascular, and help to improve the final image stitching results.  According to the analysis of Figure 3, under the condition of no transformation to be stitched, the correspondence rate of the feature point extraction results of the method in this paper reaches nearly 99%; After improving the brightness of the stitched image, the correspondence rate of the feature point extraction results of the method in this paper decreases to about 95%; After the stitched image rotates to the left, the correspondence rate of the feature point extraction results of the method in this paper decreases slightly, reaching about 92.5%; After adding Gaussian noise and magnifying the stitched image, the feature point extraction results of the method in this paper are 92% and 97% respectively; After synthesizing the above four transformations, the feature point extraction result of the method in this paper are consistently higher than 90%, while the correspondence rate of feature point extraction results from the other two methods is always lower than 90%. The above statistical results fully show that the proposed method has strong robustness for feature point extraction. Table 1 shows the image registration results obtained by the method in this paper under different conditions. According to Table 1, using the present method, the root mean square error of image registration results is lower than 2.692 under different conditions, which can realize image registration at a high registration Angle.

image stitching results
The method in this paper is used to stitch the collected consultation images of fundus vascular of patients. The results are shown in Figure 4 and Figure 5.   Remote consultation image stitching method based on wireless sensor technology and mathematical morphology 9 The method in this paper can not only reduce the visual field of the retina without the seam of Figure 4, but also get an excellent visual effect on the basis of image stitching. In order to further verify the excellent effect of the method in this paper on image stitching, the left eye image is taken as an example, the method based on convolution neural network in reference [6] and the method based on homography matrix compensation in reference [7] are taken as the comparison methods, and the two comparison methods are used to splice the images respectively. The results are shown in Figure 6.   Figure 6, it can be seen that the seam in the stitched image obtained by stitching with comparison method 1 is obvious (circular marked area); The seam in the stitched image obtained by stitching with comparison method 2 is more significant. Comparing Figure 4 (b) and Figure 6, we can see that the method in this paper has better stitching effect. This is because the filter reduces the noise on the accuracy, improves the accuracy of consultation image acquisition, reduces the distortion of image signal, reduces the life cycle, uses SURF algorithm to construct the scale space, determines the main direction of the feature points in the image and improves the accuracy of the image.

Conclusion
This paper studies the remote consultation image stitching method based on wireless sensor technology and mathematical morphology. The remote consultation image is collected by wireless sensor technology, and the filter is designed based on mathematical morphology to improve the image acquisition accuracy. Based on the image acquisition results, the image stitching method based on feature point matching is used to complete the remote consultation image stitching. The test results show that this method has good stitching performance. However, although this method solves the browsing and analysis problems of large field of view and high-power images, the analysis speed of large field of view and highpower images containing a huge amount of data and the evaluation method of analysis results are still unsatisfactory, which is a problem to be further solved in the future.