Abstract
Histopathological image recognition of breast cancer is an onerous task. Although many deep learning models have achieved good classification results on histopathological image classification tasks, these models do not take full advantage of the staining properties of histopathological images. In this paper, we propose a novel Deconv-Transformer (DecT) network model, which incorporates the color deconvolution in the form of convolution layers. This model uses a self-attention mechanism to match the independent properties of the HED channel information obtained by the color deconvolution. It also uses a method similar to the residual connection to fuse the information of both RGB and HED color space images, which can compensate for the information loss in the process of transferring RGB images to HED images. The training process of the DecT model is divided into two stages so that the parameters of the deconvolution layer can be better adapted to different types of histopathological images. We use the color jitter in the image data augmentation process to reduce the overfitting in the model training process. The DecT model achieves an average accuracy of 93.02% and F1-score of 0.9389 on BreakHis dataset, and an average accuracy of 79.06% and 81.36% on BACH and UC datasets.