Comparison of Segmentation Analysis in Nucleus Detection with GLCM Features using Otsu and Polynomial Methods

Pap smear is a digital image generated from the recording of cervical cancer cell preparation. Images generated are susceptible to errors due to the relatively small cell sizes and overlapping cell nuclei. Therefore, accurate Pap smear image analysis is essential to obtain the right information. This research compares nucleus segmentation and detection using Grey Level Co-occurrence Matrix (GLCM) features in two methods: Otsu and Polynomial. The tested data consisted of 400 images sourced from RepoMedUNM, a publicly accessible repository containing 2,346 images. Both methods were compared and evaluated to obtain the most accurate features. The research results showed that the average distance of the Otsu method was 6.6457, which was superior to the Polynomial method with a value of 6.6215. Distance refers to the distance between the nucleus detected by the Otsu and the Polynomial method. Distance is an important measure to assess how closely the detection results align with the actual nucleus positions. It indicates that the Polynomial method produces nucleus detections that are on average closer to the actual nucleus positions compared to the Otsu method.  Consequently, this research can serve as a reference for further studies in developing new methods to enhance the accuracy of identification.