TwinNet: A Symmetrically Structured Convolutional Architecture for Identifying Tibetan Characters

Abstract Although the Tibetan language is widely used, its intelligent application is seriously lagging behind. Most studies on character recognition have almost ignored minority languages like Tibetan. For the purpose of recognizing Tibetan characters, a convolutional architecture named TwinNet is proposed in this work. Specifically, two parallel convolutional sub-networks sharing the same parameters were firstly carefully designed and connected using an energy function, thus achieving Tibetan character recognition via one-shot verification task based on similarity metric. Second, a fuzzy c-means clustering module based on statistical laws of Tibetan characters was integrated into the pipeline of TwinNet, thus greatly reducing the search space at the recognition stage. Third, a Tibetan similar character dataset (TSCD) is constructed after sufficient amount of mining and analysis work, providing data support for training supervised models. The results of binary classification experiment demonstrate that TwinNet can differentiate the similar and dissimilar image pairs with a recall of 0.92, a precision of 0.89, an accuracy of 0.90, an F-1 score of 0.90 and a Kappa coefficient of 0.82. Furthermore, the consistency between the output TwinNet and the subjective evaluation by Tibetan-speaking volunteers is also evaluated. The experimental results support the idea that the similarity evaluation of Tibetan characters by TwinNet is highly consistent with that of humans, the indicator of SROCC, PLCC and RMSE are 0.68, 0.84 and 0.04, respectively. The results of character recognition experiments based on one-shot verification task show that the improved model with the integration of the fuzzy c-means clustering module has an average accuracy of 92% for regular characters and 80% for severely deformed characters.