A Review of Convolutional Neural Network Development in Computer Vision
DOI:
https://doi.org/10.4108/eetiot.v7i28.445Keywords:
Convolutional Neural Networks, computer vision, deep learning, IoTAbstract
Convolutional neural networks have made admirable progress in computer vision. As a fast-growing computer field, CNNs are one of the classical and widely used network structures. The Internet of Things (IoT) has gotten a lot of attention in recent years. This has directly led to the vigorous development of AI technology, such as the intelligent luggage security inspection system developed by the IoT, intelligent fire alarm system, driverless car, drone technology, and other cutting-edge directions. This paper first outlines the structure of CNNs, including the convolutional layer, the downsampling layer, and the fully connected layer, all of which play an important role. Then some different modules of classical networks are described, and these modules are rapidly driving the development of CNNs. And then the current state of CNNs research in image classification, object segmentation, and object detection is discussed.
Downloads
References
Y. Sun, X. Wang, and X. Tang, "Deeply learned face representations are sparse, selective, and robust," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 2892-2900.
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, vol. 25, 2012.
M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," in European conference on computer vision, 2014: Springer, pp. 818-833.
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:1312.4400, 2013.
C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
S. Srinivas and R. V. Babu, "Data-free parameter pruning for deep neural networks," arXiv preprint arXiv:1507.06149, 2015.
X. Liu, J. Pool, S. Han, and W. J. Dally, "Efficient sparse-winograd convolutional neural networks," arXiv preprint arXiv:1802.06367, 2018.
M. Jaderberg, A. Vedaldi, and A. Zisserman, "Speeding up convolutional neural networks with low rank expansions," arXiv preprint arXiv:1405.3866, 2014.
H. Zhou, J. M. Alvarez, and F. Porikli, "Less is more: Towards compact cnns," in European conference on computer vision, 2016: Springer, pp. 662-677.
E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, "Exploiting linear structure within convolutional networks for efficient evaluation," Advances in neural information processing systems, vol. 27, 2014.
W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, "Learning structured sparsity in deep neural networks," Advances in neural information processing systems, vol. 29, 2016.
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, "A survey of convolutional neural networks: analysis, applications, and prospects," IEEE Transactions on Neural Networks and Learning Systems, 2021.
C. Hwang, D. Kim, and T. Lee, "Semi-supervised based Unknown Attack Detection in EDR Environment," KSII Transactions on Internet and Information Systems, vol. 14, no. 12, pp. 4909-4926, Dec 2020.
H. Jung and B. G. Lee, "The Impact of Transforming Unstructured Data into Structured Data on a Churn Prediction Model for Loan Customers," KSII Transactions on Internet and Information Systems, vol. 14, no. 12, pp. 4706-4724, Dec 2020.
V. Lebedev and V. Lempitsky, "Fast convnets using group-wise brain damage," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2554-2564.
Y.-D. Lv, "Alcoholism detection by data augmentation and convolutional neural network with stochastic pooling," Journal of Medical Systems, vol. 42, no. 1, 2018, Art no. 2.
C. Tang, "Twelve-layer deep convolutional neural network with stochastic pooling for tea category classification on GPU platform," Multimedia Tools and Applications, vol. 77, no. 17, pp. 22821-22839, 2018.
C. Pan, "Abnormal breast identification by nine-layer convolutional neural network with parametric rectified linear unit and rank-based stochastic pooling," Journal of Computational Science, vol. 27, pp. 57-68, 2018.
N. Altwaijry, "Keystroke Dynamics Analysis for User Authentication Using a Deep Learning Approach," International Journal of Computer Science and Network Security, vol. 20, no. 12, pp. 209-216, Dec 2020.
M. Raveendra and K. Nagireddy, "Inter frame Tampering Detection based on DWT-DCT Markov Features and Fine tuned AlexNet Model," International Journal of Computer Science and Network Security, vol. 20, no. 12, pp. 1-12, Dec 2020.
T. Kattenborn, J. Leitloff, F. Schiefer, and S. Hinz, "Review on Convolutional Neural Networks (CNN) in vegetation remote sensing," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 173, pp. 24-49, 2021.
P. Murugeswari and S. Vijayalakshmi, "New Method of Internal Type-2 Fuzzy-Based CNN for Image Classification," Int. J. Fuzzy Log. Intell. Syst., vol. 20, no. 4, pp. 336-345, Dec 2020.
S. Joshi, R. Kumar, and A. Dwivedi, "Hybrid DSSCS and convolutional neural network for peripheral blood cell recognition system," IET Image Processing, vol. 14, no. 17, pp. 4450-4460, Dec 2020.
C. Pan, "Multiple sclerosis identification by convolutional neural network with dropout and parametric ReLU," Journal of Computational Science, vol. 28, pp. 1-10, 2018/09/01/ 2018.
C. Huang, "Multiple Sclerosis Identification by 14-Layer Convolutional Neural Network With Batch Normalization, Dropout, and Stochastic Pooling," (in English), Frontiers in Neuroscience, Original Research vol. 12, 2018-November-08 2018, Art no. 818.
G. Zhao, "Polarimetric synthetic aperture radar image segmentation by convolutional neural network using graphical processing units," Journal of Real-Time Image Processing, vol. 15, no. 3, pp. 631-642, 2018.
C. Vance et al., "Learning to detect the onset of slow activity after a generalized tonic-clonic seizure," Bmc Medical Informatics and Decision Making, vol. 20, Dec 2020, Art no. 330.
H. Sim and J. Lee, "Bitstream-Based Neural Network for Scalable, Efficient, and Accurate Deep Learning Hardware," Frontiers in Neuroscience, vol. 14, Dec 2020, Art no. 543472.
H. K. Shin, S. H. Park, and K. W. Kim, "Inter-floor noise classification using convolutional neural network," Plos One, vol. 15, no. 12, Dec 2020, Art no. e0243758.
K. Muhammad, "Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation," Multimedia Tools and Applications, vol. 78, no. 3, pp. 3613-3632, 2019.
S.-H. Wang and J. Sun, "Cerebral micro-bleeding identification based on a nine-layer convolutional neural network with stochastic pooling," Concurrency and Computation: Practice and Experience, vol. 32, no. 1, p. e5130, 2020.
A. K. Sangaiah, "Alcoholism identification via convolutional neural network based on parametric ReLU, dropout, and batch normalization," Neural Computing and Applications, vol. 32, pp. 665-680, 2020.
G. Antonelli, P. Gkolfakis, G. Tziatzios, I. S. Papanikolaou, K. Triantafyllou, and C. Hassan, "Artificial intelligence-aided colonoscopy: Recent developments and future perspectives," World Journal of Gastroenterology, vol. 26, no. 47, pp. 7436-7443, Dec 2020.
R. Majji, P. G. O. Prakash, R. Cristin, and G. Parthasarathy, "Social bat optimisation dependent deep stacked auto-encoder for skin cancer detection," Iet Image Processing, vol. 14, no. 16, pp. 4122-4131, Dec 2020.
N. Padmasini and R. Umamaheswari, "Automated detection of multiple structural changes of diabetic macular oedema in SDOCT retinal images through transfer learning in CNNs," Iet Image Processing, vol. 14, no. 16, pp. 4067-4075, Dec 2020.
P. Sinthia and M. Malathi, "Cancer detection using convolutional neural network optimized by multistrategy artificial electric field algorithm," International Journal of Imaging Systems and Technology, vol. 31, no. 3, pp. 1386-1403, Sep 2021.
M. S. Yildirim and E. Dandil, "Automatic detection of multiple sclerosis lesions using Mask R-CNN on magnetic resonance scans," Iet Image Processing, vol. 14, no. 16, pp. 4277-4290, Dec 2020.
Y. D. Zhang, "A seven-layer convolutional neural network for chest CT based COVID-19 diagnosis using stochastic pooling," IEEE Sens. J., pp. 1-1. doi: 10.1109/JSEN.2020.3025855
S.-H. Wang, "Covid-19 Classification by FGCNet with Deep Feature Fusion from Graph Convolutional Network and Convolutional Neural Network," Information Fusion, vol. 67, pp. 208-229, 2020/10/09/ 2021.
Y.-D. Zhang, "A five-layer deep convolutional neural network with stochastic pooling for chest CT-based COVID-19 diagnosis," Machine Vision and Applications, vol. 32, 2021, Art no. 14.
M. Taskiran, N. Kahraman, and C. E. Erdem, "Hybrid face recognition under adverse conditions using appearance-based and dynamic features of smile expression," Iet Biometrics, vol. 10, no. 1, pp. 99-115, Jan 2021.
M. M. Rahman and F. H. Siddiqui, "Multi-layered attentional peephole convolutional LSTM for abstractive text summarization," Etri Journal, vol. 43, no. 2, pp. 288-298, Apr 2021.
P. Dey, "The emerging role of deep learning in cytology," Cytopathology, vol. 32, no. 2, pp. 154-160, Mar 2021.
D. S. Guttery, "Improved Breast Cancer Classification Through Combining Graph Convolutional Network and Convolutional Neural Network," Information Processing and Management, vol. 58, 2, 2021, Art no. 102439.
X. Cheng, "PSSPNN: PatchShuffle Stochastic Pooling Neural Network for an Explainable Diagnosis of COVID-19 with Multiple-Way Data Augmentation," Computational and Mathematical Methods in Medicine, vol. 2021, 2021, Art no. 6633755.
W. Zhu, "ANC: Attention Network for COVID-19 Explainable Diagnosis Based on Convolutional Block Attention Module," Computer Modeling in Engineering & Sciences, vol. 127, 3, pp. 1037-1058, 2021.
C. Liu, J. Yuen, and A. Torralba, "Nonparametric scene parsing via label transfer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2368-2382, 2011.
K. Kiwitz, C. Schiffer, H. Spitzer, T. Dickscheid, and K. Amunts, "Deep learning networks reflect cytoarchitectonic features used in brain mapping," Scientific Reports, vol. 10, no. 1, Dec 2020, Art no. 22039.
A. S. Nencka et al., "Split-slice training and hyperparameter tuning of RAKI networks for simultaneous multi-slice reconstruction," Magn. Reson. Med., p. 9. doi: 10.1002/mrm.28634 Article; Early Access. [Online]. Available: ://WOS:000599191800001
E. Kotze and B. Senekal, "Not just a language with white faces: Analysing #taalmonument on Instagram using machine learning," Td-the Journal for Transdisciplinary Research in Southern Africa, vol. 16, no. 1, Dec 2020, Art no. a871.
D. Marima, B. Hadad, S. Froim, A. Eyal, and A. Bahabad, "Visual data detection through side-scattering in a multimode optical fiber," Opt. Lett., vol. 45, no. 24, pp. 6724-6727, Dec 2020.
Y. Pang, M. Sun, X. Jiang, and X. Li, "Convolution in convolution for network in network," IEEE transactions on neural networks and learning systems, vol. 29, no. 5, pp. 1587-1597, 2017.
S. Zagoruyko and N. Komodakis, "Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer," arXiv preprint arXiv:1612.03928, 2016.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929-1958, 2014.
J. Yim, D. Joo, J. Bae, and J. Kim, "A gift from knowledge distillation: Fast optimization, network minimization and transfer learning," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4133-4141.
R. K. Pandey and R. A. Ganesan, "DeepInterpolation: fusion of multiple interpolations and CNN to obtain super-resolution," Iet Image Processing, vol. 14, no. 15, pp. 4000-4011, Dec 2020.
Q. Zhou, "ADVIAN: Alzheimer's Disease VGG-Inspired Attention Network Based on Convolutional Block Attention Module and Multiple Way Data Augmentation," Front. Aging Neurosci., vol. 13, 2021, Art no. 687456.
S. C. Satapathy and D. Wu, "Improving ductal carcinoma in situ classification by convolutional neural network with exponential linear unit and rank-based weighted pooling," Complex Intell. Syst., vol. 7, pp. 1295-1310, 2020/11/22 2021.
R. Girshick, F. Iandola, T. Darrell, and J. Malik, "Deformable part models are convolutional neural networks," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2015, pp. 437-446.
C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in European conference on computer vision, 2014: Springer, pp. 184-199.
J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1646-1654.
Y. T. Xiao, "TReC: Transferred ResNet and CBAM for Detecting Brain Diseases," Front. Neuroinformatics, vol. 15, Dec 2021, Art no. 781551.
S. Lu, "Detecting pathological brain via ResNet and randomized neural networks," Heliyon, vol. 6, no. 12, p. e05625, 2020.
M. Mora, J. Naranjo-Torres, and V. Aubin, "Convolutional Neural Networks for Off-Line Writer Identification Based on Simple Graphemes," Applied Sciences-Basel, vol. 10, no. 22, Nov 2020, Art no. 7999.
R. K. Srivastava, K. Greff, and J. Schmidhuber, "Highway networks," arXiv preprint arXiv:1505.00387, 2015.
F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251-1258.
G. Larsson, M. Maire, and G. Shakhnarovich, "Fractalnet: Ultra-deep neural networks without residuals," arXiv preprint arXiv:1605.07648, 2016.
J. Cheng, P.-s. Wang, G. Li, Q.-h. Hu, and H.-q. Lu, "Recent advances in efficient computation of deep convolutional neural networks," Frontiers of Information Technology & Electronic Engineering, vol. 19, no. 1, pp. 64-77, 2018.
Y. Cheng, D. Wang, P. Zhou, and T. Zhang, "A survey of model compression and acceleration for deep neural networks," arXiv preprint arXiv:1710.09282, 2017.
S. Zagoruyko and N. Komodakis, "Wide residual networks," arXiv preprint arXiv:1605.07146, 2016.
S. Targ, D. Almeida, and K. Lyman, "Resnet in resnet: Generalizing residual architectures," arXiv preprint arXiv:1603.08029, 2016.
X. Zhang, X. Zhou, M. Lin, and J. Sun, "Shufflenet: An extremely efficient convolutional neural network for mobile devices," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6848-6856.
K. Zhang, M. Sun, T. X. Han, X. Yuan, L. Guo, and T. Liu, "Residual networks of residual networks: Multilevel residual networks," IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 6, pp. 1303-1314, 2017.
M. Abdi and S. Nahavandi, "Multi-residual networks: Improving the speed and accuracy of residual networks," arXiv preprint arXiv:1609.05672, 2016.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708.
S.-H. Wang, "DenseNet-201-Based Deep Neural Network with Composite Learning Factor and Precomputation for Multiple Sclerosis Classification," ACM Trans. Multimedia Comput. Commun. Appl., vol. 16, no. 2s, p. Article 60, 2020.
S. C. Satapathy, "Covid-19 diagnosis via DenseNet and optimization of transfer learning setting," Cognitive Computation. doi: 10.1007/s12559-020-09776-8
M. Astaraki, G. Yang, Y. Zakko, I. Toma-Dasu, O. Smedby, and C. L. Wang, "A Comparative Study of Radiomics and Deep-Learning Based Methods for Pulmonary Nodule Malignancy Prediction in Low Dose CT Images," Frontiers in Oncology, vol. 11, Dec 2021, Art no. 737368.
H. S. Shad et al., "Comparative Analysis of Deepfake Image Detection Method Using Convolutional Neural Network," Computational Intelligence and Neuroscience, vol. 2021, Dec 2021, Art no. 3111676.
D. Sulot, D. Alonso-Caneiro, D. R. Iskander, and M. J. Collins, "Deep learning approaches for segmenting Bruch's membrane opening from OCT volumes," OSA Continuum, vol. 3, no. 12, pp. 3351-3364, Dec 2020.
A. Woloshuk et al., "In Situ Classification of Cell Types in Human Kidney Tissue Using 3D Nuclear Staining," Cytometry Part A, vol. 99, no. 7, pp. 707-721, Jul 2021.
K. Wu, "SOSPCNN: Structurally Optimized Stochastic Pooling Convolutional Neural Network for Tetralogy of Fallot Recognition," Wireless Communications and Mobile Computing, vol. 2021, p. 5792975, 2021/07/02 2021, Art no. 5792975.
G. Andrew and Z. Menglong, "Efficient convolutional neural networks for mobile vision applications," ed: Mobilenets, 2017.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "Mobilenetv2: Inverted residuals and linear bottlenecks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510-4520.
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
Y. Sun, X. Wang, and X. Tang, "Deeply learned face representations are sparse, selective, and robust," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 2892-2900. DOI: https://doi.org/10.1109/CVPR.2015.7298907
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998. DOI: https://doi.org/10.1109/5.726791
A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, vol. 25, 2012.
M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," in European conference on computer vision, 2014: Springer, pp. 818-833. DOI: https://doi.org/10.1007/978-3-319-10590-1_53
K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:1312.4400, 2013.
C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9. DOI: https://doi.org/10.1109/CVPR.2015.7298594
K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778. DOI: https://doi.org/10.1109/CVPR.2016.90
S. Srinivas and R. V. Babu, "Data-free parameter pruning for deep neural networks," arXiv preprint arXiv:1507.06149, 2015. DOI: https://doi.org/10.5244/C.29.31
X. Liu, J. Pool, S. Han, and W. J. Dally, "Efficient sparse-winograd convolutional neural networks," arXiv preprint arXiv:1802.06367, 2018.
M. Jaderberg, A. Vedaldi, and A. Zisserman, "Speeding up convolutional neural networks with low rank expansions," arXiv preprint arXiv:1405.3866, 2014. DOI: https://doi.org/10.5244/C.28.88
H. Zhou, J. M. Alvarez, and F. Porikli, "Less is more: Towards compact cnns," in European conference on computer vision, 2016: Springer, pp. 662-677. DOI: https://doi.org/10.1007/978-3-319-46493-0_40
E. L. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, "Exploiting linear structure within convolutional networks for efficient evaluation," Advances in neural information processing systems, vol. 27, 2014.
W. Wen, C. Wu, Y. Wang, Y. Chen, and H. Li, "Learning structured sparsity in deep neural networks," Advances in neural information processing systems, vol. 29, 2016.
Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, "A survey of convolutional neural networks: analysis, applications, and prospects," IEEE Transactions on Neural Networks and Learning Systems, 2021. DOI: https://doi.org/10.1109/TNNLS.2021.3084827
C. Hwang, D. Kim, and T. Lee, "Semi-supervised based Unknown Attack Detection in EDR Environment," KSII Transactions on Internet and Information Systems, vol. 14, no. 12, pp. 4909-4926, Dec 2020.
H. Jung and B. G. Lee, "The Impact of Transforming Unstructured Data into Structured Data on a Churn Prediction Model for Loan Customers," KSII Transactions on Internet and Information Systems, vol. 14, no. 12, pp. 4706-4724, Dec 2020.
V. Lebedev and V. Lempitsky, "Fast convnets using group-wise brain damage," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2554-2564. DOI: https://doi.org/10.1109/CVPR.2016.280
Y.-D. Lv, "Alcoholism detection by data augmentation and convolutional neural network with stochastic pooling," Journal of Medical Systems, vol. 42, no. 1, 2018, Art no. 2. DOI: https://doi.org/10.1007/s10916-017-0845-x
C. Tang, "Twelve-layer deep convolutional neural network with stochastic pooling for tea category classification on GPU platform," Multimedia Tools and Applications, vol. 77, no. 17, pp. 22821-22839, 2018. DOI: https://doi.org/10.1007/s11042-018-5765-3
C. Pan, "Abnormal breast identification by nine-layer convolutional neural network with parametric rectified linear unit and rank-based stochastic pooling," Journal of Computational Science, vol. 27, pp. 57-68, 2018. DOI: https://doi.org/10.1016/j.jocs.2018.05.005
N. Altwaijry, "Keystroke Dynamics Analysis for User Authentication Using a Deep Learning Approach," International Journal of Computer Science and Network Security, vol. 20, no. 12, pp. 209-216, Dec 2020.
M. Raveendra and K. Nagireddy, "Inter frame Tampering Detection based on DWT-DCT Markov Features and Fine tuned AlexNet Model," International Journal of Computer Science and Network Security, vol. 20, no. 12, pp. 1-12, Dec 2020.
T. Kattenborn, J. Leitloff, F. Schiefer, and S. Hinz, "Review on Convolutional Neural Networks (CNN) in vegetation remote sensing," ISPRS Journal of Photogrammetry and Remote Sensing, vol. 173, pp. 24-49, 2021. DOI: https://doi.org/10.1016/j.isprsjprs.2020.12.010
P. Murugeswari and S. Vijayalakshmi, "New Method of Internal Type-2 Fuzzy-Based CNN for Image Classification," Int. J. Fuzzy Log. Intell. Syst., vol. 20, no. 4, pp. 336-345, Dec 2020. DOI: https://doi.org/10.5391/IJFIS.2020.20.4.336
S. Joshi, R. Kumar, and A. Dwivedi, "Hybrid DSSCS and convolutional neural network for peripheral blood cell recognition system," IET Image Processing, vol. 14, no. 17, pp. 4450-4460, Dec 2020. DOI: https://doi.org/10.1049/iet-ipr.2020.0370
C. Pan, "Multiple sclerosis identification by convolutional neural network with dropout and parametric ReLU," Journal of Computational Science, vol. 28, pp. 1-10, 2018/09/01/ 2018. DOI: https://doi.org/10.1016/j.jocs.2018.07.003
C. Huang, "Multiple Sclerosis Identification by 14-Layer Convolutional Neural Network With Batch Normalization, Dropout, and Stochastic Pooling," (in English), Frontiers in Neuroscience, Original Research vol. 12, 2018-November-08 2018, Art no. 818. DOI: https://doi.org/10.3389/fnins.2018.00818
G. Zhao, "Polarimetric synthetic aperture radar image segmentation by convolutional neural network using graphical processing units," Journal of Real-Time Image Processing, vol. 15, no. 3, pp. 631-642, 2018. DOI: https://doi.org/10.1007/s11554-017-0717-0
C. Vance et al., "Learning to detect the onset of slow activity after a generalized tonic-clonic seizure," Bmc Medical Informatics and Decision Making, vol. 20, Dec 2020, Art no. 330. DOI: https://doi.org/10.1186/s12911-020-01308-6
H. Sim and J. Lee, "Bitstream-Based Neural Network for Scalable, Efficient, and Accurate Deep Learning Hardware," Frontiers in Neuroscience, vol. 14, Dec 2020, Art no. 543472. DOI: https://doi.org/10.3389/fnins.2020.543472
H. K. Shin, S. H. Park, and K. W. Kim, "Inter-floor noise classification using convolutional neural network," Plos One, vol. 15, no. 12, Dec 2020, Art no. e0243758. DOI: https://doi.org/10.1371/journal.pone.0243758
K. Muhammad, "Image based fruit category classification by 13-layer deep convolutional neural network and data augmentation," Multimedia Tools and Applications, vol. 78, no. 3, pp. 3613-3632, 2019. DOI: https://doi.org/10.1007/s11042-017-5243-3
S.-H. Wang and J. Sun, "Cerebral micro-bleeding identification based on a nine-layer convolutional neural network with stochastic pooling," Concurrency and Computation: Practice and Experience, vol. 32, no. 1, p. e5130, 2020. DOI: https://doi.org/10.1002/cpe.5130
A. K. Sangaiah, "Alcoholism identification via convolutional neural network based on parametric ReLU, dropout, and batch normalization," Neural Computing and Applications, vol. 32, pp. 665-680, 2020. DOI: https://doi.org/10.1007/s00521-018-3924-0
G. Antonelli, P. Gkolfakis, G. Tziatzios, I. S. Papanikolaou, K. Triantafyllou, and C. Hassan, "Artificial intelligence-aided colonoscopy: Recent developments and future perspectives," World Journal of Gastroenterology, vol. 26, no. 47, pp. 7436-7443, Dec 2020. DOI: https://doi.org/10.3748/wjg.v26.i47.7436
R. Majji, P. G. O. Prakash, R. Cristin, and G. Parthasarathy, "Social bat optimisation dependent deep stacked auto-encoder for skin cancer detection," Iet Image Processing, vol. 14, no. 16, pp. 4122-4131, Dec 2020. DOI: https://doi.org/10.1049/iet-ipr.2020.0318
N. Padmasini and R. Umamaheswari, "Automated detection of multiple structural changes of diabetic macular oedema in SDOCT retinal images through transfer learning in CNNs," Iet Image Processing, vol. 14, no. 16, pp. 4067-4075, Dec 2020. DOI: https://doi.org/10.1049/iet-ipr.2020.0612
P. Sinthia and M. Malathi, "Cancer detection using convolutional neural network optimized by multistrategy artificial electric field algorithm," International Journal of Imaging Systems and Technology, vol. 31, no. 3, pp. 1386-1403, Sep 2021. DOI: https://doi.org/10.1002/ima.22530
M. S. Yildirim and E. Dandil, "Automatic detection of multiple sclerosis lesions using Mask R-CNN on magnetic resonance scans," Iet Image Processing, vol. 14, no. 16, pp. 4277-4290, Dec 2020. DOI: https://doi.org/10.1049/iet-ipr.2020.1128
Y. D. Zhang, "A seven-layer convolutional neural network for chest CT based COVID-19 diagnosis using stochastic pooling," IEEE Sens. J., pp. 1-1. doi: 10.1109/JSEN.2020.3025855 DOI: https://doi.org/10.1109/JSEN.2020.3025855
S.-H. Wang, "Covid-19 Classification by FGCNet with Deep Feature Fusion from Graph Convolutional Network and Convolutional Neural Network," Information Fusion, vol. 67, pp. 208-229, 2020/10/09/ 2021. DOI: https://doi.org/10.1016/j.inffus.2020.10.004
Y.-D. Zhang, "A five-layer deep convolutional neural network with stochastic pooling for chest CT-based COVID-19 diagnosis," Machine Vision and Applications, vol. 32, 2021, Art no. 14. DOI: https://doi.org/10.1007/s00138-020-01128-8
M. Taskiran, N. Kahraman, and C. E. Erdem, "Hybrid face recognition under adverse conditions using appearance-based and dynamic features of smile expression," Iet Biometrics, vol. 10, no. 1, pp. 99-115, Jan 2021. DOI: https://doi.org/10.1049/bme2.12006
M. M. Rahman and F. H. Siddiqui, "Multi-layered attentional peephole convolutional LSTM for abstractive text summarization," Etri Journal, vol. 43, no. 2, pp. 288-298, Apr 2021. DOI: https://doi.org/10.4218/etrij.2019-0016
P. Dey, "The emerging role of deep learning in cytology," Cytopathology, vol. 32, no. 2, pp. 154-160, Mar 2021. DOI: https://doi.org/10.1111/cyt.12942
D. S. Guttery, "Improved Breast Cancer Classification Through Combining Graph Convolutional Network and Convolutional Neural Network," Information Processing and Management, vol. 58, 2, 2021, Art no. 102439. DOI: https://doi.org/10.1016/j.ipm.2020.102439
X. Cheng, "PSSPNN: PatchShuffle Stochastic Pooling Neural Network for an Explainable Diagnosis of COVID-19 with Multiple-Way Data Augmentation," Computational and Mathematical Methods in Medicine, vol. 2021, 2021, Art no. 6633755. DOI: https://doi.org/10.1155/2021/6633755
W. Zhu, "ANC: Attention Network for COVID-19 Explainable Diagnosis Based on Convolutional Block Attention Module," Computer Modeling in Engineering & Sciences, vol. 127, 3, pp. 1037-1058, 2021. DOI: https://doi.org/10.32604/cmes.2021.015807
C. Liu, J. Yuen, and A. Torralba, "Nonparametric scene parsing via label transfer," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2368-2382, 2011. DOI: https://doi.org/10.1109/TPAMI.2011.131
K. Kiwitz, C. Schiffer, H. Spitzer, T. Dickscheid, and K. Amunts, "Deep learning networks reflect cytoarchitectonic features used in brain mapping," Scientific Reports, vol. 10, no. 1, Dec 2020, Art no. 22039. DOI: https://doi.org/10.1038/s41598-020-78638-y
A. S. Nencka et al., "Split-slice training and hyperparameter tuning of RAKI networks for simultaneous multi-slice reconstruction," Magn. Reson. Med., p. 9. doi: 10.1002/mrm.28634 Article; Early Access. [Online]. Available: ://WOS:000599191800001
E. Kotze and B. Senekal, "Not just a language with white faces: Analysing #taalmonument on Instagram using machine learning," Td-the Journal for Transdisciplinary Research in Southern Africa, vol. 16, no. 1, Dec 2020, Art no. a871. DOI: https://doi.org/10.4102/td.v16i1.871
D. Marima, B. Hadad, S. Froim, A. Eyal, and A. Bahabad, "Visual data detection through side-scattering in a multimode optical fiber," Opt. Lett., vol. 45, no. 24, pp. 6724-6727, Dec 2020. DOI: https://doi.org/10.1364/OL.408552
Y. Pang, M. Sun, X. Jiang, and X. Li, "Convolution in convolution for network in network," IEEE transactions on neural networks and learning systems, vol. 29, no. 5, pp. 1587-1597, 2017. DOI: https://doi.org/10.1109/TNNLS.2017.2676130
S. Zagoruyko and N. Komodakis, "Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer," arXiv preprint arXiv:1612.03928, 2016.
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," The journal of machine learning research, vol. 15, no. 1, pp. 1929-1958, 2014.
J. Yim, D. Joo, J. Bae, and J. Kim, "A gift from knowledge distillation: Fast optimization, network minimization and transfer learning," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4133-4141. DOI: https://doi.org/10.1109/CVPR.2017.754
R. K. Pandey and R. A. Ganesan, "DeepInterpolation: fusion of multiple interpolations and CNN to obtain super-resolution," Iet Image Processing, vol. 14, no. 15, pp. 4000-4011, Dec 2020. DOI: https://doi.org/10.1049/iet-ipr.2019.1244
Q. Zhou, "ADVIAN: Alzheimer's Disease VGG-Inspired Attention Network Based on Convolutional Block Attention Module and Multiple Way Data Augmentation," Front. Aging Neurosci., vol. 13, 2021, Art no. 687456. DOI: https://doi.org/10.3389/fnagi.2021.687456
S. C. Satapathy and D. Wu, "Improving ductal carcinoma in situ classification by convolutional neural network with exponential linear unit and rank-based weighted pooling," Complex Intell. Syst., vol. 7, pp. 1295-1310, 2020/11/22 2021. DOI: https://doi.org/10.1007/s40747-020-00218-4
R. Girshick, F. Iandola, T. Darrell, and J. Malik, "Deformable part models are convolutional neural networks," in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2015, pp. 437-446. DOI: https://doi.org/10.1109/CVPR.2015.7298641
C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in European conference on computer vision, 2014: Springer, pp. 184-199. DOI: https://doi.org/10.1007/978-3-319-10593-2_13
J. Kim, J. K. Lee, and K. M. Lee, "Accurate image super-resolution using very deep convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1646-1654. DOI: https://doi.org/10.1109/CVPR.2016.182
Y. T. Xiao, "TReC: Transferred ResNet and CBAM for Detecting Brain Diseases," Front. Neuroinformatics, vol. 15, Dec 2021, Art no. 781551. DOI: https://doi.org/10.3389/fninf.2021.781551
S. Lu, "Detecting pathological brain via ResNet and randomized neural networks," Heliyon, vol. 6, no. 12, p. e05625, 2020. DOI: https://doi.org/10.1016/j.heliyon.2020.e05625
M. Mora, J. Naranjo-Torres, and V. Aubin, "Convolutional Neural Networks for Off-Line Writer Identification Based on Simple Graphemes," Applied Sciences-Basel, vol. 10, no. 22, Nov 2020, Art no. 7999. DOI: https://doi.org/10.3390/app10227999
R. K. Srivastava, K. Greff, and J. Schmidhuber, "Highway networks," arXiv preprint arXiv:1505.00387, 2015.
F. Chollet, "Xception: Deep learning with depthwise separable convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251-1258. DOI: https://doi.org/10.1109/CVPR.2017.195
G. Larsson, M. Maire, and G. Shakhnarovich, "Fractalnet: Ultra-deep neural networks without residuals," arXiv preprint arXiv:1605.07648, 2016.
J. Cheng, P.-s. Wang, G. Li, Q.-h. Hu, and H.-q. Lu, "Recent advances in efficient computation of deep convolutional neural networks," Frontiers of Information Technology & Electronic Engineering, vol. 19, no. 1, pp. 64-77, 2018. DOI: https://doi.org/10.1631/FITEE.1700789
Y. Cheng, D. Wang, P. Zhou, and T. Zhang, "A survey of model compression and acceleration for deep neural networks," arXiv preprint arXiv:1710.09282, 2017.
S. Zagoruyko and N. Komodakis, "Wide residual networks," arXiv preprint arXiv:1605.07146, 2016. DOI: https://doi.org/10.5244/C.30.87
S. Targ, D. Almeida, and K. Lyman, "Resnet in resnet: Generalizing residual architectures," arXiv preprint arXiv:1603.08029, 2016.
X. Zhang, X. Zhou, M. Lin, and J. Sun, "Shufflenet: An extremely efficient convolutional neural network for mobile devices," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 6848-6856. DOI: https://doi.org/10.1109/CVPR.2018.00716
K. Zhang, M. Sun, T. X. Han, X. Yuan, L. Guo, and T. Liu, "Residual networks of residual networks: Multilevel residual networks," IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 6, pp. 1303-1314, 2017. DOI: https://doi.org/10.1109/TCSVT.2017.2654543
M. Abdi and S. Nahavandi, "Multi-residual networks: Improving the speed and accuracy of residual networks," arXiv preprint arXiv:1609.05672, 2016.
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, "Densely connected convolutional networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700-4708. DOI: https://doi.org/10.1109/CVPR.2017.243
S.-H. Wang, "DenseNet-201-Based Deep Neural Network with Composite Learning Factor and Precomputation for Multiple Sclerosis Classification," ACM Trans. Multimedia Comput. Commun. Appl., vol. 16, no. 2s, p. Article 60, 2020. DOI: https://doi.org/10.1145/3341095
S. C. Satapathy, "Covid-19 diagnosis via DenseNet and optimization of transfer learning setting," Cognitive Computation. doi: 10.1007/s12559-020-09776-8 DOI: https://doi.org/10.1007/s12559-020-09776-8
M. Astaraki, G. Yang, Y. Zakko, I. Toma-Dasu, O. Smedby, and C. L. Wang, "A Comparative Study of Radiomics and Deep-Learning Based Methods for Pulmonary Nodule Malignancy Prediction in Low Dose CT Images," Frontiers in Oncology, vol. 11, Dec 2021, Art no. 737368. DOI: https://doi.org/10.3389/fonc.2021.737368
H. S. Shad et al., "Comparative Analysis of Deepfake Image Detection Method Using Convolutional Neural Network," Computational Intelligence and Neuroscience, vol. 2021, Dec 2021, Art no. 3111676. DOI: https://doi.org/10.1155/2021/3111676
D. Sulot, D. Alonso-Caneiro, D. R. Iskander, and M. J. Collins, "Deep learning approaches for segmenting Bruch's membrane opening from OCT volumes," OSA Continuum, vol. 3, no. 12, pp. 3351-3364, Dec 2020. DOI: https://doi.org/10.1364/OSAC.403102
A. Woloshuk et al., "In Situ Classification of Cell Types in Human Kidney Tissue Using 3D Nuclear Staining," Cytometry Part A, vol. 99, no. 7, pp. 707-721, Jul 2021. DOI: https://doi.org/10.1002/cyto.a.24274
K. Wu, "SOSPCNN: Structurally Optimized Stochastic Pooling Convolutional Neural Network for Tetralogy of Fallot Recognition," Wireless Communications and Mobile Computing, vol. 2021, p. 5792975, 2021/07/02 2021, Art no. 5792975.
G. Andrew and Z. Menglong, "Efficient convolutional neural networks for mobile vision applications," ed: Mobilenets, 2017.
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, "Mobilenetv2: Inverted residuals and linear bottlenecks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4510-4520. DOI: https://doi.org/10.1109/CVPR.2018.00474
P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "Overfeat: Integrated recognition, localization and detection using convolutional networks," arXiv preprint arXiv:1312.6229, 2013.
X. Han and Q. Dai, "Batch-normalized Mlpconv-wise supervised pre-training network in network," Applied Intelligence, vol. 48, no. 1, pp. 142-155, 2018. DOI: https://doi.org/10.1007/s10489-017-0968-2
D. T. Nguyen, W. Li, and P. O. Ogunbona, "Human detection from images and videos: A survey," Pattern Recognition, vol. 51, pp. 148-175, 2016. DOI: https://doi.org/10.1016/j.patcog.2015.08.027
Y. Li, S. Wang, Q. Tian, and X. Ding, "Feature representation for statistical-learning-based object detection: A review," Pattern Recognition, vol. 48, no. 11, pp. 3542-3559, 2015. DOI: https://doi.org/10.1016/j.patcog.2015.04.018
M. Pedersoli, A. Vedaldi, J. Gonzalez, and X. Roca, "A coarse-to-fine approach for fast deformable object detection," Pattern Recognition, vol. 48, no. 5, pp. 1844-1853, 2015. DOI: https://doi.org/10.1016/j.patcog.2014.11.006
S. Zagoruyko et al., "A multipath network for object detection," arXiv preprint arXiv:1604.02135, 2016. DOI: https://doi.org/10.5244/C.30.15
P. N. Sabes and M. I. Jordan, "Advances in neural information processing systems," in In G. Tesauro & D. Touretzky & T. Leed (Eds.), Advances in Neural Information Processing Systems, 1995: Citeseer.
S. Hong, T. You, S. Kwak, and B. Han, "Online tracking by learning discriminative saliency map with convolutional neural network," in International conference on machine learning, 2015: PMLR, pp. 597-606.
J. Fan, W. Xu, Y. Wu, and Y. Gong, "Human tracking using convolutional neural networks," IEEE transactions on Neural Networks, vol. 21, no. 10, pp. 1610-1623, 2010. DOI: https://doi.org/10.1109/TNN.2010.2066286
R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 580-587. DOI: https://doi.org/10.1109/CVPR.2014.81
R. Girshick, "Fast r-cnn," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1440-1448. DOI: https://doi.org/10.1109/ICCV.2015.169
S. Ren, K. He, R. Girshick, and J. Sun, "Faster r-cnn: Towards real-time object detection with region proposal networks," Advances in neural information processing systems, vol. 28, 2015.
T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature pyramid networks for object detection," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117-2125. DOI: https://doi.org/10.1109/CVPR.2017.106
K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask r-cnn," in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961-2969. DOI: https://doi.org/10.1109/ICCV.2017.322
E. Shelhamer, J. Long, and T. Darrell, "Fully convolutional networks for semantic segmentation," IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 4, pp. 640-651, 2016. DOI: https://doi.org/10.1109/TPAMI.2016.2572683
M. E. Basiri, S. Nemati, M. Abdar, E. Cambria, and U. R. Acharya, "ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis," Future Generation Computer Systems, vol. 115, pp. 279-294, 2021. DOI: https://doi.org/10.1016/j.future.2020.08.005
M. Torres and F. Cantú, "Learning to see: Convolutional neural networks for the analysis of social science data," Political Analysis, vol. 30, no. 1, pp. 113-131, 2022. DOI: https://doi.org/10.1017/pan.2021.9
D. Sarvamangala and R. V. Kulkarni, "Convolutional neural networks in medical image understanding: a survey," Evolutionary intelligence, pp. 1-22, 2021. DOI: https://doi.org/10.1007/s12065-020-00540-3
A.-A. Tulbure, A.-A. Tulbure, and E.-H. Dulf, "A review on modern defect detection models using DCNNs–Deep convolutional neural networks," Journal of Advanced Research, vol. 35, pp. 33-48, 2022. DOI: https://doi.org/10.1016/j.jare.2021.03.015
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 EAI Endorsed Transactions on Internet of Things
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
This is an open-access article distributed under the terms of the Creative Commons Attribution CC BY 3.0 license, which permits unlimited use, distribution, and reproduction in any medium so long as the original work is properly cited.
