Facial expression recognition in the wild is challenging due to various unconstrained conditions. Although existing facial expression classifiers have been almost perfect on analyzing constrained frontal faces, they fail to perform well on partially occluded faces that are common in the wild. To avoid those challenges in recent work this paper deals with the facial expression detection and recognition through Viola-Jones algorithm and HCNN using LSTM method. For face detection, basically utilize the face detection Viola-Jones algorithm and it recognizes the occluded face and it helps to perform the feature selection through the whale optimization algorithm, once after compression and further, it minimizes the feature vector given into the Hybrid Convolutional Neural Network (HCNN) and Long Short-Term Memory (LSTM) model for identifying the facial expression in efficient manner. In existing work, feature extraction stage exact finding of the corner becomes very difficult task. To solve this issue, we need to use corner detection algorithm to enhance the feature extraction (corner points) from the face image. One of the main drawbacks of WOA is that it is not good at exploring the search space. To overcome those issues this work introduced an improved framework for facial image recognition, in which first edges are detected using Canny edge detection operator. Then improved linear contrast stretching is used for image enhancement. Then in feature extraction, we propose hybrid Scale-Invariant Feature Transform (SIFT) with double d-LBP (Dd-LBP) to obtain the features that are illumination and pose independent. For face detection, we utilize the face detection Viola-Jones algorithm and it recognizes the occluded face and significant features are selected by using self-learning chicken swarm optimization, it minimizes the feature vector given into the Hybrid Convolutional Neural Network (HCNN) and Long Short-Term Memory (LSTM) model for identifying the facial expression in efficient manner. Experimental results demonstrate the efficiency of the proposed work in terms of accuracy, precision, recall and f-measure.

Viola-Jones algorithm, face detection, edge detection, feature matching, artificial bee colony, hybrid convolutional neural network.

Received: February 18, 2021; Accepted: March 20, 2021; Published: March 24, 2021

How to cite this article: Dinesh Kumar P and B. Rosiline Jeetha, Canny Edge Detection and Contrast Stretching for Facial Expression Detection and Recognition Using Machine Learning, Far East Journal of Electronics and Communications 24(1) (2021), 35-66. DOI: 10.17654/EC024010035

This Open Access Article is Licensed under Creative Commons Attribution 4.0 International License

References:

[1] N. Zeng, H. Zhang, B. Song, W. Liu, Y. Li and A. M. Dobaie, Facial expression recognition via learning deep sparse auto encoders, Neuro Computing 273 (2018), 643-649.
[2] N. B. Kar, K. S. Babu and S. K. Jena, Face expression recognition using histograms of oriented gradients with reduced features, Proceedings of International Conference on Computer Vision and Image Processing, Springer, Singapore, 2017, pp. 209-219.
[3] S. Li and W. Deng, Reliable crowd sourcing and deep locality-preserving learning for unconstrained facial expression recognition, IEEE Transactions on Image Processing 28(1) (2018), 356-370.
[4] K. Zhang, Y. Huang, Y. Du and L. Wang, Facial expression recognition based on deep evolutional spatial-temporal networks, IEEE Transactions on Image Processing 26(9) (2017), 4193-4203.
[5] K. Shan, J. Guo, W. You, D. Lu and R. Bie, Automatic facial expression recognition based on a deep convolutional-neural-network structure, 2017 IEEE 15th International Conference on Software Engineering Research, Management and Applications (SERA), IEEE, 2017, pp. 123-128.
[6] K. Wang, X. Peng, J. Yang, D. Meng and Y. Qiao, Region attention networks for pose and occlusion robust facial expression recognition, IEEE Transactions on Image Processing 29 (2020), pp. 4057-4069.
[7] S. Nigam, R. Singh and A. K. Misra, Efficient facial expression recognition using histogram of oriented gradients in wavelet domain, Multimedia Tools and Applications, 77(21) (2018), 28725-28747.
[8] Y. H. Lai and S. H. Lai, Emotion-preserving representation learning via generative adversarial network for multi-view facial expression recognition, 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), 2018, pp. 263-270.
[9] J. C. Batista, V. Albiero, O. R. Bellon and L. Silva, Aumpnet: simultaneous action units detection and intensity estimation on multipose facial images using a single convolutional neural network, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), 2017, pp. 866-871.
[10] E. Sariyanidi, H. Gunes and A. Cavallaro, Learning bases of activity for facial expression recognition, IEEE Transactions on Image Processing 26(4) (2017), 1965-1978.
[11] S. Kumar, S. Singh and J. Kumar, Automatic live facial expression detection using genetic algorithm with haar wavelet features and SVM, Wireless Personal Communications 103(3) (2018), 2435-2453.
[12] I. Gogić, M. Manhart, I. S. Pandžić and J. Ahlberg, Fast facial expression recognition using local binary features and shallow neural networks, The Visual Computer 36(1) (2020), 97-112.
[13] H. Yang, Z. Zhang and L. Yin, Identity-adaptive facial expression recognition through expression regeneration using conditional generative adversarial networks, In 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), 2018, pp. 294-301.
[14] Y. Ding, Q. Zhao, B. Li and X. Yuan, Facial expression recognition from image sequence based on LBP and Taylor expansion, IEEE Access 5 (2017), 19409-19419.
[15] S. K. A. Kamarol, M. H. Jaward, H. Kälviäinen, J. Parkkinen and R. Parthiban, Joint facial expression recognition and intensity estimation based on weighted votes of image sequences, Pattern Recognition Letters 92 (2017), 25-32.
[16] J. H. Shah, M. Sharif, M. Yasmin and S. L. Fernandes, Facial expressions classification and false label reduction using LDA and threefold SVM, Pattern Recognition Letters, 2017.
[17] C. Qi, M. Li, Q. Wang, H. Zhang, J. Xing, Z. Gao and H. Zhang, Facial expressions recognition based on cognition and mapped binary patterns, IEEE Access, 6 (2018), 18795-18803.
[18] G. Shi, J. Suo, C. Liu, K. Wan and X. Lv, October. Moving target detection algorithm in image sequences based on edge detection and frame difference, 2017 IEEE 3rd Information Technology and Mechatronics Engineering Conference (ITOEC), 2017, pp. 740-744.
[19] K. Gaurav and U. Ghanekar, Image steganography based on Canny edge detection, dilation operator and hybrid coding, Journal of Information Security and Applications 41 (2018), 41-51.
[20] G. Deng and Y. Wu, Double lane line edge detection method based on constraint conditions Hough transform, 2018 17th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES), 2018, pp. 107-110.
[21] S. Bhairannawar, A. Patil, A. Janmane and M. Huilgol, Color image enhancement using Laplacian filter and contrast limited adaptive histogram equalization, 2017 Innovations in Power and Advanced Computing Technologies (i-PACT), 2017, pp. 1-5.
[22] Z. Shi, Y. Feng, M. Zhao, E. Zhang and L. He, Let you see in sand dust weather: A method based on halo-reduced dark channel prior dehazing for sand-dust image enhancement, IEEE Access 7 (2019), 116722-116733.
[23] A. Mustapha, A. Oulefki, M. Bengherabi, E. Boutellaa and M. A. Algaet, Towards nonuniform illumination face enhancement via adaptive contrast stretching, Multimedia Tools and Applications 76(21) (2017), 21961-21999.
[24] S. Deb, X. Z. Gao, K. Tammi, K. Kalita and P. Mahanta, Recent studies on chicken swarm optimization algorithm: a review (2014-2018), Artificial Intelligence Review, 2019, pp. 1-29.
[25] J. Wang, Z. Cheng, O. K. Ersoy, M. Zhang, K. Sun and Y. Bi, Improvement and application of chicken swarm optimization for constrained optimization, IEEE Access 7 (2019), 58053-58072.