Volume 40 Issue 1
Feb.  2022
Turn off MathJax
Article Contents
Article Navigation >  Journal of Transport Information and Safety  > 2022  >  40(1): 63-71
WANG Peng, SHEN Helong, YIN Yong, LYU Hongguang. A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique[J]. Journal of Transport Information and Safety, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008
Citation: WANG Peng, SHEN Helong, YIN Yong, LYU Hongguang. A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique[J]. Journal of Transport Information and Safety, 2022, 40(1): 63-71. doi: 10.3963/j.jssn.1674-4861.2022.01.008
shu

A Detection Algorithm for the Fatigue of Ship Officers Based on Deep Learning Technique

doi: 10.3963/j.jssn.1674-4861.2022.01.008
  • 1.

    Key Laboratory of Marine Simulation & Control for Ministry of Transportation, Dalian Maritime Univers ity, Dalian 116026, Liaoning, China

  • 2.

    Navigation College, Dalian Maritime University, Dalian 116026, Liaoning, China

  • Received Date: 2021-08-12
    Available Online: 2022-03-31
    Abstract
  • Aiming at preventing Officers on Watch (OOW) from fatigue driving, a fatigue detection and alert algorithm based on deep learning technique is developed. Considering the large space and complex background of the ship bridge, the RetinaFace model is improved by using Depthwise Separable Convolution to optimize the detection speed. An upgraded ShuffleNetV2 network is then developed by adopting the concepts of Channel Split, Channel Shuffle, and other techniques such as batch normalization and global average pooling. The proposed algorithm can extract image features and automatically identify the opening and closing of the eyes and mouth of the OOW. According to the PERCLOS criteria, the two features of the eyes and mouth are integrated to determine whether the OOW is fatigued. Experimental results show that the detection speed of the improved RetinaFace model improves from 9.33 to 22.60 frames/s. The detection accuracy and speed for the face detection are superior to the multi-task convolutional neural network. The upgraded ShuffleNetV2 network achieves over 99.50% accuracy in recognizing the states of eyes and mouth. The algorithm has an accuracy of 95.70% and a recall rate of 96.73% in identifying the fatigue state in a simulated ship bridge scenario, which are higher than Haar-like+Adaboost and MTCNN+CNN fatigue detection algorithmsused in practice. It only takes 0.083 s for the algorithm to complete the process, which indicates that the algorithm is capable of carrying out real-time detection.

     

    • FullText(HTML)
  • loading
    • References(20)
  • [1]
    郭其顺. 世界海难事故的现状与原因分析[J]. 中国船检, 2000(3): 44-46. https://www.cnki.com.cn/Article/CJFDTOTAL-ZGCJ200003030.htm

    GUO Q S. Current situation and cause analysis of maritime accidents in the world[J]. China Ship Survey, 2000(3): 44-46. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-ZGCJ200003030.htm
    [2]
    张堃. 船舶机舱值班人员眼部视觉识别及疲劳状态监测[D]. 大连: 大连海事大学, 2019.

    ZHANG K. Eye visual recognition and fatigue monitoring of engineering watch-keepers on board ship[D]. Dalian: Dalian Maritime University, 2019. (in Chinese)
    [3]
    刘建蓓, 马小龙, 张志伟, 等. 基于心电分析的青藏高原驾驶人疲劳特性[J]. 交通运输工程学报, 2016, 16(4): 151-158. doi: 10.3969/j.issn.1671-1637.2016.04.016

    LIU J B, MA X L, ZHANG Z W, et al. Fatigue characteristics of driver in Qinghai-Tibet Plateau based on electrocardiogram analysis[J]. Journal of Traffic and Transportation Engineering, 2016, 16(4): 151-158. (in Chinese) doi: 10.3969/j.issn.1671-1637.2016.04.016
    [4]
    LAWOYIN S, FEI D Y, BAI O. Accelerometer-based steering-wheel movement monitoring for drowsy-driving detection[J]. Journal of automobile engineering, 2015, 229(2): 163-173. doi: 10.1177/0954407014536148
    [5]
    刘梦佳, 赵建国. 基于SVM的疲劳驾驶人脸图像特征检测算法[J]. 半导体光电, 2020, 41(2): 278-282. https://www.cnki.com.cn/Article/CJFDTOTAL-BDTG202002026.htm

    LIU M J, ZHAO J G. Research on feature detection algorithm of fatigue driving face image based on SVM[J]. Semiconductor Optoelectronics, 2020, 41(2): 278-282. (in Chinese) https://www.cnki.com.cn/Article/CJFDTOTAL-BDTG202002026.htm
    [6]
    王迪. 基于人眼状态的疲劳检测算法研究与应用[D]. 成都: 电子科技大学, 2020.

    WANG D. Research and application of fatigue detection algorithm based on human eye state[D]. Chengdu: University of Electronic Science and Technology of China, 2020. (in Chinese)
    [7]
    黄玲, 洪佩鑫, 吴泽荣, 等. 基于防疫常态化的驾驶员疲劳状态检测方法[J]. 交通信息与安全, 2021, 39(4): 26-34. doi: 10.3963/j.jssn.1674-4861.2021.04.004

    HUANG L, HONG P X, WU Z R, et al. A detection method for drivers' fatigue states based on normalization of epidemic prevention[J]. Journal of Transport Information and Safety, 2021, 39(4): 26-34. (in Chinese) doi: 10.3963/j.jssn.1674-4861.2021.04.004
    [8]
    张博雅. 基于深度学习的驾驶安全评估方法研究[D]. 西安: 西安科技大学, 2019.

    ZHANG B Y. Research on driving safety assessment method based on deep learing[D]. Xi'an: Xi'an University Of Science And Technology, 2019. (in Chinese)
    [9]
    LIU Z M, PENG Y X, HU W J. Driver fatigue detection based on deeply-learned facial expression representation[J]. Journal of Visual Communication and Image Representation, 2020 (71): 102-107.
    [10]
    郭佳伟. 基于计算机视觉的驾驶员异常行为识别与预警[D]. 大连: 大连海事大学, 2019.

    GUO J W. Driver's abnormal behavior recognition and early-warning based on computation vision[D]. Dalian: Dalian Maritime University, 2019. (in Chinese)
    [11]
    LI K N, GONG Y B, REN Z L. A fatigue driving detection algorithm based on facial multi-feature fusion[J]. IEEE Access, 2020, 8(11): 244-259.
    [12]
    陈军, 张黎, 周博, 等. 基于级联卷积神经网络的驾驶员分心驾驶行为检测[J]. 科学技术与工程, 2020, 20(14): 5702-5708. doi: 10.3969/j.issn.1671-1815.2020.14.037

    CHEN J, ZHANG L, ZHOU B, et al. Driver distracted driving behavior detection based on cascaded convolutional neural network[J]. Science Technology and Engineering, 2020, 20(14): 5702-5708. (in Chinese) doi: 10.3969/j.issn.1671-1815.2020.14.037
    [13]
    DENG J K, GUO J, VERVERAS E, et al. Retinaface: Single-shot multi-level face localisation in the wild[C]. The IEEE conference on computer vision and pattern recognition, Seattle: IEEE, 2020.
    [14]
    GAVAI N R, JAKHADE Y A, TRIBHUVAN S A, et al. MobileNets for flower classification using TensorFlow[C]. 2017 International Conference on Big Data, IoT and Data Science (BID), Pune: IEEE, 2017.
    [15]
    YANG S, LUO P, LOY C C, et al. Wider face: A face detection benchmark[C]. The IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas: IEEE, 2016.
    [16]
    ZHANG K P, ZHANG Z P, LI Z F, et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters, 2016, 23(10): 1499-1503. doi: 10.1109/LSP.2016.2603342
    [17]
    MA N N, ZHANG X Y, ZHENG H T, et al. ShuffleNet V2: practical guidelines for efficient CNN architecture design[C]. European Conference on Computer Vision, Munich: ECCV, 2018.
    [18]
    IOFFE S, SZEGEDY C. Batch normalization: Accelerating deep network training by reducing internal covariate Shift[C]. International Conference on Machine Learning, Lille: PMLR, 2015.
    [19]
    PENG Y, DONG Y, CHENG D. Design and implementation of a driver's eye state recognition algorithm based on PERCLOS[J]. Chinese Journal of Electronics, 2014, 23(4): 669-672.
    [20]
    ABTAHI S, OMIDYEGANEH M, SHIRMOHAMMADI S, et al. YawDD: A yawning detection dataset[C]. The 5th ACM multimedia systems conference, Singapore: ACM, 2014.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(12)  / Tables(2)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views (1355) PDF downloads(130) Cited by()
    Proportional views
    Related

    Copyright Editorial Office of Transport Information and Safety鄂ICP备05003336号-2

    Address：No. 1178，Heping Avenue, Wuchang, Wuhan, CHINA (430063)

    Tel：027-86580355 Email：jtjsj@vip.163.com

    Supported by: Beijing Renhe Information Technology Co. Ltd  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return