ResNet-based dairy daily behavior recognition

With the rapid development of China ’s dairy farming industry, it becomes difficult to breed and manage the increasing number of dairy cows. The smart agricultural enabled by edging techniques such as smart sensor, IoT, machine learning, etc. shows great potential to improve the scientific breeding and management of dairy cows. Using machine learning assisted computer vision to identify and classify the behavior of dairy cows can quickly determine the health status of dairy cows and improve management efficiency. However, there are still some challenges need to be addressed in the current behavior recognition of dairy cows. Due to the more complex background of dairy farms, the increase in the number of cows makes the mutual shading problem of dairy cows serious, which leads to the low efficiency of dairy cow behavior recognition. To address this challenge, this paper collected and labeled four types of 1,660 dairy daily behavior datasets and proposed a residual neural network (ResNet)-based dairy daily behavior recognition model. Experiments show that the proposed method is far superior to the baseline method in accuracy performance, and it provides inspiration for the behavior recognition of cows.


Introduction
The basis of dairy industry development is dairy cows. In recent years, the number of dairy cows in our country has increased rapidly, and the prospects for economic benefits are good. However, the production of individual dairy cows in China's local dairy cows is small, and it cannot bear the burden of large-scale industrial production of milk. Become an important part of the development of China's dairy industry. The dramatic increase in the number of dairy cows has brought challenges to the breeding and management of dairy cows. With the rapid development of computer vision, behavior recognition has been used more and more in dairy farming. The daily behaviors, estrus behaviors, and body protection behaviors of dairy cows have been taken as research priorities. Recognizing the daily behavior of dairy cows is a challenging task. First of all, you need to define the behavior of the cow. In addition * Corresponding author. Email:lifuzhong@sxau.edu.cn to the basic behavior of the cow, such as standing, walking, running, lying, and ruminating, you need to define some special behaviors of the cow, such as looking back, eating, eating, crawling, chasing, and then These behaviors are classified as normal and abnormal behaviors, and serve as a standard for judging whether cows are normal or not. Secondly, due to the current large-scale free-range cows, the background of the background dairy farm is complex, and the noise of the picture is very large when the behavior is recognized. The increase in the number of cows will often cause occlusion problems, making it difficult to identify the behavior of the cows.
In view of the above problems, this paper proposes a behavior recognition model based on ResNet by studying the behavior of cows. Compared with traditional methods, this method has the advantages of small manual workload, fast recognition speed, and small memory consumption. Good results have been achieved in the dataset, and we hope that this method can be beneficial to behavior recognition applications and research in related fields. L. J. Cheng et al. 2 The organizational structure of this article is as follows: Section 2 provides a literature and analyzed the advantages and disadvantages of traditional behavior recognition research methods; the Section 3 introduces the construction of dairy cow daily behavior datasets and the characteristics of the datasets; the third chapter elaborates the ResNet network in detail Model construction, experiment, results; Section 4 analyzes the results of the experiment in detail; Section 5 concludes to future work.

Related works
There are three general methods for the behavior recognition of dairy cows. The first one is animal behavior research based on wearable sensors. By wearing 3D sensors and collar detectors for cows, they can monitor the lying, standing, ruminating, feeding, Normal and lame, you can conduct specific research on a behavior or action, antiinterference is strong, and there are no restrictions on the angle and distance of video collection, but the information taken by the sensor is relatively simple and the cost is high. And wearing the sensor will interfere with the normal life behavior of the cow to a certain extent. Not suitable for large-scale farms.
The second method is the traditional method in behavior recognition. First, target detection is performed. After training with a large amount of labeled data, the main body of the cow can be identified, and the target area is extracted from the image. Secondly, extract the features of the detection target, get the shape, position, color and other information of the moving target from the image, so as to find the position of the moving target in the image. Finally, the establishment of the model, the main classification models used are SVM classifier, BP neural network classification model, KNN classification model. Because the traditional method provides more effective information, the computer can accurately and quickly find the points of interest that need attention, so the recognition accuracy is higher. In particular, the lying behavior is significantly different from other behaviors, with an accuracy rate of 100%, a minimum walking recognition rate of 95.85%, and an average correct recognition rate of 97.32% [1]. However, the manual manual labeling required in the previous work is heavy, the time required to train the classification model is long, and the memory is large. This method is the current mainstream cow behavior recognition method.
The third method is to use neural networks to identify the behavior of cows. The neural network uses layers to train the parameters of each layer. After the first layer is calculated, the neurons will pass the data to the second layer, and the second layer of neurons will perform the tasks. , And so on until the last layer, and then output the result. This step can be regarded as an unsupervised learning process. This is the biggest difference from the traditional method, also known as feature learning. Therefore, the neural network requires less manual labeling. The feature search depends on the calculation of the computer itself, and the memory occupied by the operation is small and the speed is faster. The neural network constructed by the batch normalization method can also effectively improve the network training speed [2]. However, because the judgment of the neural task is a "probability vector" which is actually based on the guess results given by the weights, the accuracy rate will be lower than that of the traditional method. However, the neural network has a small workload and higher work efficiency and can better adapt to the processing of massive data on large-scale dairy farms. Commonly used models include ResNet, GeogLeNet, RNN, etc.
Inspired by the above methods, we propose to identify the basic behavior of dairy cows through neural networks, and select GeogLeNet, ResNet18, ResNet50 three networks for comparative analysis experiments.

Construction of dairy cow daily behavior dataset
Because the video processing speed is slow, and it takes up a lot of memory, and the video processing is essentially to perform optical flow recognition on the pictures of adjacent frames in the video, we give priority to starting with pictures to solve the problem of behavior recognition. The data collection mainly uses the Hikvision 5 million home outdoor remote mobile phone poe surveillance highdefinition camera DS-2CD3T25-I3, a total of four. One Hikvision video recorder and one hard disk. The hard disk is 3T, and data is backed up and formatted every week. Taking Holstein cows in the pasture station of Shanxi Agricultural University as the main research object, through high-definition cameras, from September 1, 2019, to January 1, 2019, 90 cows in the cowshed were captured 24 hours a day to collect data.
According to the collected data, the main direction is to distinguish the special behaviors of cows before and after calving. Figure 1. shows our classification of cow behavior is mainly divided into four categories: standing, walking, lying, and looking back.

Figure 1. Partial dataset diagram
Among them, standing, walking, and lying are the normal behaviors of cows, while looking back is a special ResNet-based dairy daily behavior recognition 3 behavior. The pre-natal behavior of cows is a reaction of physiological and psychological behaviors under specific conditions. It generally shows typical behaviors such as frequent waking up and looking back. Therefore, it is extremely important for cows to obtain information on the behavior and regularity of the cow's behavior during this period [2]. The method of video collection is mainly fixed imaging equipment, which is collected from the top, side and rear of the cow's playground. Finally, after processing about 10T of video, 1663 databases of cow behaviors including 4 behaviors were constructed for the next experiment, Table 1. shows the behavior classification diagram. The location where we selected to collect data is the cow's sports ground, which is the main activity place for cows. About 24 hours in 24 hours are in the sports ground. The environment of the sports ground is more complicated, and cow dung is not cleaned on a large scale, so this dataset is more in line with the natural environment. However, when the cow adopts a lying position, the cow dung will stick to the leg, which causes the computer to judge the accuracy of the cow's movement. Figure 2. shows the shielding problem between the cows and the indistinguishability from the background environment make it difficult to identify the cow's behavior.

ResNet-based cow behavior recognition model Network structure
ResNet is divided into 5 models, the main difference is the residual block block and the number of layers, the block has two (Bottleneck/Basicblock), different models call different classes. In Resnet50, resnet101, resnet152 call the Bottleneck class, and in Resnet18 and resnet34 call the BasicBlock class. Therefore, we chose ResNet18 and ResNet50 for experiments and analysis. The network structure is as shown in Table 2. In ResNet50, softmax has a total of 50 layers. In stage1, there are 3 Bottlenecks. The final output image size is 256*56*56, and the step size is 1. There are 4 Bottleneck in stage2, the output image size is 512*28*28, and the step size is adjusted to 2. There are 6 Bottleecks in stage3, the output image size is 1024*14*14, and the step size is 2. Stage4 has three Bottlenecks, the output image size is 2048*7*7, and the step size is 2. The overall network structure is as follows in Figure 3. Data preprocessing and data enhancement Based on the original 1663 data, in order to improve the generalization and robustness of the model.In order to avoid overfitting, we performed data enhancement, rotated the image, and modified the corresponding xml file, the rotation angle was 90, 120, 150, and finally obtained 1055 data samples, which together with the original sample constitute 2718 The picture serves as our final training and test data. In this paper, the data is divided into training set and test set, accounting for 70% and 30% of the total sample, 1903 training set samples, 815 test set samples. The training set is used for model training, and the test set is used for model prediction and evaluation. In order to enable the network to train more quickly and accurately, the dataset is normalized, and the attribute data is scaled to make the dataset uniformly become 224×224.

Experimental results and analysis
The experiment was conducted in GPU mode, using 16GB memory, 33MHz processor, dual Road TAITAN X graphics card. Dependency libraries are: numpy, matpol, sklearn, keras, etc.
Model training and testing During model training, our input picture size is 224*224, batch size is 16, learn The learning rate is 0.001, and stochastic gradient descent (SGD) is used for training optimization. Decay 1e-6 weights are used. Dropout is set to 0.5. Also adopted batch standardization technology. A total of 90 training cycles were iterated. As for the initialization of weights, we used the kaiming initialization method. At the same time, in order to monitor the training and testing process, TensorBoard is used. To have an intuitive understanding of the training process.
Model evaluation indicators In general, we can evaluate the neural network through some indicators and change it through evaluation Into our neural network. The methods for evaluating neural networks and the methods for evaluating machine learning are similar, and common ones include errors, accuracy, and R2 score. This article uses the general accuracy and loss function to evaluate the effectiveness of our model. In addition to ResNet, we also chose GeogLeNet Inception V3 for comparison between different networks. The experimental results are as follows in both Table 3 and Figure 4.

Figure 4. Experimental results
Judging from the final experimental results, the ResNet50 model is superior to GeogLeNet in accuracy and loss rate, and ResNet50 is superior to ResNet18, so it can be concluded that the proposed method for daily behavior recognition of dairy cows based on ResNet50 is It has good effect, higher accuracy than Baseline, and has the advantages of high efficiency and fast speed and has broad application prospects. From the perspective of the accuracy of each type of cow movement, the behavior of lying down is obviously different from other behaviors, and the accuracy rate is the highest among the three networks. In the action of looking back, the analysis of the wrongly recognized pictures shows that because the constructed dataset is a picture, the look back is a dynamic behavior of the cow. If the action is completed, the characteristics are obvious and it is easier to identify, but if the action is performed at the beginning or the end, it is more difficult to distinguish from the standing movement of the cow. The low recognition accuracy of walking and standing is because the cow's front and back standing and the cow's front and back walking, the difference between the two types of actions is low. Figure 5. shows the dairy behavior recognition accuracy rate confusion matrix.

Conclusion and Future works
ResNet 50 is excellent in the behavior recognition of cows, but as a network model for image recognition, it is weak for some behaviors with dynamic and temporal characteristics. Specifically in this experiment, the lying behavior recognition accuracy rate is the highest, and the remaining three types of movements, standing, walking, and looking back still have room for improvement. Moreover, when faced with a complex background, the accuracy rate will decrease. But the recognition speed is faster. Compared with the traditional methods used by other scholars, although the accuracy rate is reduced, the neural network requires less manual operations and faster time, so the method in this paper has a lot of room for development.
In the next work, the experiment will perform some processing on the complex background, for example, use Mask-RCNN to process the image, segment the cow body from the complex background, in order to reduce noise, by establishing such a based on ResNet50 and Mask-RCNN's hybrid model attempts to improve the shortcomings of this method, improve the accuracy of dairy cows' daily behavior identification, so as to predict the health of dairy cows more accurately, and facilitate the management and breeding of cattle farms.

Acknowledgements.
Throughout the writing of this dissertation I have received a great deal of support and assistance.
I would first like to thank my dean, Li fuzhong, and my colleague,Zhang wuping,whose expertise was invaluable in formulating the research questions and methodology.
I would like to acknowledge my team members, Jing chao and Duan tonghao for their wonderful collaboration and patient support.
In addition, I would like to thank my family for their wise counsel and sympathetic ear. You are always there for me.
Finally, I could not have completed this dissertation without the support of my friends, who provided stimulating discussions as well as happy distractions to rest my mind outside of my research.