Skip to main content
Springer Nature Link
Log in

A Temporal Recognition Framework for Multi-sheep Behaviour Using ViTSORT and YOLOv8-MS

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15043))

Included in the following conference series:

  • 117 Accesses

Abstract

Sheep behaviour can reflect their growth and health. However, the current sheep behaviour recognition studies have yet to focus on statistics on the duration of various behaviours for individual sheep. Traditional behavioural recognition methods can overlook abnormal sheep behaviour such as prolonged lying or not eating. Therefore, we propose an advanced framework for statistically analyzing the duration of sheep behaviours within a farm environment. This paper constructed a dataset of sheep behaviour images collected from a natural farm environment, including walking, standing, eating, lame, lying, licking, and attacking. Based on the Vision Transformer (ViT) method and the YOLOv8 model, a sheep tracking model, ViTSORT, and a sheep behaviour recognition model, YOLOv8-MS, are presented for the duration of sheep behaviours. The experimental results show that ViTSORT can solve the problem of tracking target loss when sheep cover each other. Meanwhile, the YOLOv8-MS model achieves a precision of 96.9% mAP on our sheep behaviour dataset, and the detection speed is 196 FPS, higher than previous methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
EUR 29.95
Price includes VAT (France)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
EUR 70.18
Price includes VAT (France)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
EUR 86.50
Price includes VAT (France)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Arcidiacono, C., Porto, S., Mancino, M., Cascone, G.: Development of a threshold-based classifier for real-time recognition of cow feeding and standing behavioural activities from accelerometer data. Comput. Electron. Agric. 134, 124–134 (2017)

    Article  Google Scholar 

  2. Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 3464–3468. IEEE (2016)

    Google Scholar 

  3. Chelotti, J.O., Vanrell, S.R., Rau, L.S.M., Galli, J.R., Planisich, A.M., Utsumi, S.A., Milone, D.H., Giovanini, L.L., Rufiner, H.L.: An online method for estimating grazing and rumination bouts using acoustic signals in grazing cattle. Comput. Electron. Agric. 173, 105443 (2020)

    Article  Google Scholar 

  4. Chen, C., Zhu, W., Norton, T.: Behaviour recognition of pigs and cattle: Journey from computer vision to deep learning. Comput. Electron. Agric. 187, 106255 (2021)

    Article  Google Scholar 

  5. Chen, C., Zhu, W., Steibel, J., Siegford, J., Han, J., Norton, T.: Classification of drinking and drinker-playing in pigs by a video-based deep learning method. Biosys. Eng. 196, 1–14 (2020)

    Article  Google Scholar 

  6. Fuentes, A., Yoon, S., Park, J., Park, D.S.: Deep learning-based hierarchical cattle behavior recognition with spatio-temporal information. Comput. Electron. Agric. 177, 105627 (2020)

    Article  MATH  Google Scholar 

  7. Gao, Y., Yan, K., Dai, B., Sun, H., Yin, Y., Liu, R., Shen, W.: Recognition of aggressive behavior of group-housed pigs based on cnn-gru hybrid model with spatio-temporal attention mechanism. Comput. Electron. Agric. 205, 107606 (2023)

    Article  Google Scholar 

  8. Gu, Z., Zhang, H., He, Z., Niu, K.: A two-stage recognition method based on deep learning for sheep behavior. Comput. Electron. Agric. 212, 108143 (2023)

    Article  Google Scholar 

  9. Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C.: Ghostnet: More features from cheap operations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1580–1589 (2020)

    Google Scholar 

  10. Hou, Q., Jiang, Z., Yuan, L., Cheng, M.M., Yan, S., Feng, J.: Vision permutator: a permutable mlp-like architecture for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 1328–1334 (2022)

    Article  MATH  Google Scholar 

  11. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: efficient convolutional neural networks for mobile vision applications (2017). arXiv preprint arXiv:1704.04861

  12. Jiang, B., Yin, X., Song, H.: Single-stream long-term optical flow convolution network for action recognition of lameness dairy cow. Comput. Electron. Agric. 175, 105536 (2020)

    Article  MATH  Google Scholar 

  13. Li, C., Li, L., Jiang, H., Weng, K., Geng, Y., Li, L., Ke, Z., Li, Q., Cheng, M., Nie, W., et al.: Yolov6: a single-stage object detection framework for industrial applications (2022). arXiv preprint arXiv:2209.02976

  14. Lin, W., Wu, Z., Chen, J., Huang, J., Jin, L.: Scale-aware modulation meet transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6015–6026 (2023)

    Google Scholar 

  15. Liu, C., Ye, H., Wang, L., Lu, S., Li, L.: Novel tracking method for the drinking behavior trajectory of pigs. Int. J. Agric. Biol. Eng. 16(6), 67–76 (2024)

    MATH  Google Scholar 

  16. Liu, D., Oczak, M., Maschat, K., Baumgartner, J., Pletzer, B., He, D., Norton, T.: A computer vision-based method for spatial-temporal action recognition of tail-biting behaviour in group-housed pigs. Biosys. Eng. 195, 27–41 (2020)

    Article  Google Scholar 

  17. Lyu, C., Zhang, W., Huang, H., Zhou, Y., Wang, Y., Liu, Y., Zhang, S., Chen, K.: Rtmdet: an empirical study of designing real-time object detectors (2022). arXiv preprint arXiv:2212.07784

  18. Meunier, B., Pradel, P., Sloth, K.H., Cirié, C., Delval, E., Mialon, M.M., Veissier, I.: Image analysis to refine measurements of dairy cow behaviour from a real-time location system. Biosys. Eng. 173, 32–44 (2018)

    Article  Google Scholar 

  19. Shane, D.D., White, B.J., Larson, R.L., Amrine, D.E., Kramer, J.L.: Probabilities of cattle participating in eating and drinking behavior when located at feeding and watering locations by a real time location system. Comput. Electron. Agric. 127, 460–466 (2016)

    Article  Google Scholar 

  20. Wang, C.Y., Bochkovskiy, A., Liao, H.Y.M.: Yolov7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7464–7475 (2023)

    Google Scholar 

  21. Wang, J., He, Z., Zheng, G., Gao, S., Zhao, K.: Development and validation of an ensemble classifier for real-time recognition of cow behavior patterns from accelerometer data and location data. PLoS ONE 13(9), e0203546 (2018)

    Article  Google Scholar 

  22. Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3645–3649. IEEE (2017)

    Google Scholar 

  23. Wu, D., Wang, Y., Han, M., Song, L., Shang, Y., Zhang, X., Song, H.: Using a CNN-LSTM for basic behaviors detection of a single dairy cow in a complex environment. Comput. Electron. Agric. 182, 106016 (2021)

    Article  MATH  Google Scholar 

  24. Wu, D., Wu, Q., Yin, X., Jiang, B., Wang, H., He, D., Song, H.: Lameness detection of dairy cows based on the yolov3 deep learning algorithm and a relative step size characteristic vector. Biosyst. Eng. 189, 150–163 (2020)

    Article  MATH  Google Scholar 

  25. Xu, T., Zhu, X.F., Wu, X.J.: Learning spatio-temporal discriminative model for affine subspace based visual object tracking. Visual Intell. 1(1), 4 (2023)

    Article  MATH  Google Scholar 

  26. Yan, P., Liu, X., Zhang, P., Lu, H.: Learning convolutional multi-level transformers for image-based person re-identification. Visual Intell. 1(1), 24 (2023)

    Article  MATH  Google Scholar 

  27. Zhang, H., Ma, Y., Wang, X., Mao, R., Wang, M.: Lightweight real-time detection model for multi-sheep abnormal behaviour based on yolov7-tiny. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4191–4196. IEEE (2023)

    Google Scholar 

  28. Zhang, X., Liu, C., Yang, D., Song, T., Ye, Y., Li, K., Song, Y.: Rfaconv: innovating spatial attention and standard convolutional operation (2023). arXiv preprint arXiv:2304.03198

  29. Zhao, W., Xu, L.: Weakly supervised target detection based on spatial attention. Visual Intell. 2(1), 2 (2024)

    Article  MATH  Google Scholar 

  30. Zheng, A., Liu, J., Wang, Z., Huang, L., Li, C., Yin, B.: Visible-infrared person re-identification via specific and shared representations learning. Visual Intell. 1(1), 29 (2023)

    Article  Google Scholar 

  31. Zheng, Z., Zhang, X., Qin, L., Yue, S., Zeng, P.: Cows’ legs tracking and lameness detection in dairy cattle using video analysis and siamese neural networks. Comput. Electron. Agric. 205, 107618 (2023)

    Article  Google Scholar 

Download references

Acknowledgements

This work was partially funded by the Construction of the National Key Research and Development Program of China (2022ZD04014), "Scientist and Engineer" team of Qin Chuang Yuan in Shaanxi Province of China (2023KXJ-109), Qinchuangyuan Project for the Team Development of Scientists and Engineers in Shaanxi Province of China (2022KXJ-67).

Author information

Authors and Affiliations

  1. College of Information Engineering, Northwest A&F University, Yangling, 712100, China

    Yinghan Shi, Qiqi Li, Guorui Wang & Meili Wang

  2. Key Laboratory of Agricultural Internet of Things, Ministry of Agriculture, Yangling, 712100, China

    Yinghan Shi, Qiqi Li, Guorui Wang & Meili Wang

  3. Shaanxi Key Laboratory of Agricultural Information Perception and Intelligent Service, Yangling, 712100, China

    Yinghan Shi, Qiqi Li, Guorui Wang & Meili Wang

Authors
  1. Yinghan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guorui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meili Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meili Wang .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Zhouchen Lin

  2. Nankai University, Tianjin, China

    Ming-Ming Cheng

  3. Chinese Academy of Sciences, Beijing, China

    Ran He

  4. Xinjiang University, Urumqi, Xinjiang, China

    Kurban Ubul

  5. Xinjiang University, Urumqi, China

    Wushouer Silamu

  6. Peking University, Beijing, China

    Hongbin Zha

  7. Tsinghua University, Beijing, China

    Jie Zhou

  8. Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shi, Y., Li, Q., Wang, G., Wang, M. (2025). A Temporal Recognition Framework for Multi-sheep Behaviour Using ViTSORT and YOLOv8-MS. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2024. Lecture Notes in Computer Science, vol 15043. Springer, Singapore. https://doi.org/10.1007/978-981-97-8493-6_15

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-8493-6_15

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-8492-9

  • Online ISBN: 978-981-97-8493-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation