Enhance Explainability of Manifold Learning

<br>As an important component of unsupervised learning, manifold <br>learning is widely used in AI and data science. However, its <br>explainability is rarely investigated compared to that of <br>supervised learning methods though there is an urgent need from <br>both AI theory and practice. In this study, we investigate the <br>explainability of manifold learning by proposing a novel degree <br>of locality preservation (DLP) approach to evaluate the <br>efficiency of manifold learning. We develop a rigorous but easily <br>implementable DLP estimation algorithm to quantify manifold <br>learning results. We estimate the DLPs of the state-of-the-art <br>manifold learning methods: t-SNE and UMAP as well as related LLE, <br>HLLE, and LTSA methods along with widely used PCA across <br>different benchmark datasets classified as low-dimensional and <br>high-dimensional data. <br><br><br>Our study provides convincing and well-founded explanations to <br>major manifold learning methods: t-SNE and UMAP in terms of their <br>DLPs. We find t-SNE generally demonstrates a better DLP than UMAP <br>and the advantage is more obvious for low-dimensional data. The <br>order of their DLPs follows t-SNE&gt;UMAP&gt;LLE&gt;HLLE/PCA/LTSA, though <br>it may have some exceptions for some high-dimensional data. This <br>work shows both t-SNE and UMAP demonstrate an embedding distance <br>amplification mechanism under the Euclidean distance. The <br>embedding distance amplification mechanism forces the latent <br>local data geometry to stand out in dimension reduction and <br>explains why t-SNE and UMAP have higher DLPs than other peers. <br>Moreover, both t-SNE and UMAP are not locally isometric under the <br>Euclidean distance. <br><br>Furthermore, our studies discover that t-SNE and UMAP embeddings <br>show similar nonlinear nature via proposed data correlation index <br>analysis. Besides, both t-SNE and UMAP embeddings show <br>larger(smaller) data variances than the original data for low <br>(high)-dimensional data. We also propose a novel variance <br>concentration ratio (VCR) to quantify high and low dimensional <br>data and unveil that the datasets with higher VCR values would <br>have higher DLPs in manifold learning. To the best of our <br>knowledge, this study is the first work about the explainability <br>of manifold learning. The proposed methods and corresponding <br>results can be also extended to other dimension reduction <br>techniques.