An ensemble model of convolution and recurrent neural network for skin disease classification

AbstractSkin cancer is one of the rapidly growing diseases in the world. Especially, millions of cases are reported every year by all types of skin cancer in America. Early detection of skin cancer using dermoscopy, the light source, and the magnification device are used to inspect the skin lesions. A dermatologist observed hypodermic structures are normally invisible. However, accurate and effective skin disease classification by humans is not straightforward and requires a long time of practice. Furthermore, it is often inaccurate and difficult to reproduce, being unable to completely use the long‐term dependence connection between specific image key features and image labels even for experienced dermatologists. Therefore, it needs to develop a computer‐aided diagnostic system for reliable skin cancer diagnosis. Classical methods focus on designing and combining hand‐craft features from input data and face vanishing or exploding of loss gradient problem, whereas the bidirectional long short term memory (BLSTM) network does not need any prior knowledge or pre‐designing, and it is an expert in keeping the associated information in both directions. Thus, to improve the classification performance for handling these problems, we proposed a hybrid classification method based on the deep convolutional neural network and stacked BLSTM network. Firstly, deep features are extracted from input skin disease facial images. Next, the sequential features among input data are learned using a dual BLSTM network, where dual BLSTM through max‐pooling, the forward and backward long short term memory (LSTM) hidden states of both the feature matrix and its transpose concatenates for inputting into a dense, fully connected (FC) layer. Finally, the softmax function is used to classify skin disease images. To improve the generalization capability, we evaluate our method on two skin disease image datasets and compare their local image descriptors. The proposed method achieved the best mean accuracy of 91.73%, which shows significant improvements in skin disease classification compared with state‐of‐the‐art skin disease classification methods.