Persistent Homology for Structural and Overfitting Detection in Transformer  Models

PRANALI D. BHAISARE; S. B. KISHOR

Persistent Homology for Structural and Overfitting Detection in Transformer Models

Author(s)	PRANALI D. BHAISARE, Dr. S. B. KISHOR
Country	India
Abstract	Transformer models are basic and important to modern systems of natural language processing. They provide high dimensional embeddings that learn meaningful relations in the process of training. Traditional evaluation metrics like the training-validation loss gap, can find the problem of overfitting; but these embeddings avoid the evolving geometric structure of it. In this study, we used topological tool, a Persistent Homology to dissect the topological characteristics of Transformer embeddings throughout training. In this application we can constructing filtrations from embedding distances for track the birth and persistence of features such as connected components (0D) and loops (1D) These are demonstrated through the use of persistence diagrams and Betti curves. The analysis shows that overfitting leads to short-lived, unstable topologies and this is an indicator of excessive adaptiveness to noise using the one hand on the other hand, wellgeneralised models have stable and long-lived topological structures. This model-agnostic technique can detect overfitting, and offers multi-scale insights to the learning process, and is also a complementary technique to traditional diagnostic metrics. Accordingly, Topological Data Analysis is a newly emerging field that holds great potential to improve the interpretability and robustness of Deep Learning Architectures.
Keywords	Topology, Persistent, Topological Data Analysis, Homology, Betti curve, Overfitting
Field	Mathematics
Published In	Volume 17, Issue 2, April-June 2026
Published On	2026-04-09

View / Download PDF File

About IJSAT Fees & Payment Current Issue Publication Archive	Submit Research Paper Track Submission Status Publication Guidelines Publication Ethics Peer Review & Plagiarism	Join as a Reviewer Editors & Reviewers Reviewer Referral Program Get Reviewer Membership Certi.	Website/Journal Policies Usage Policy Content Policies Privacy Policy

Contact Us	Message on WhatsApp	+91-9687-182-185	editor@ijsat.org

International Journal on Science and Technology

A Widely Indexed Open Access Peer Reviewed Multidisciplinary Bi-monthly Scholarly International Journal

Persistent Homology for Structural and Overfitting Detection in Transformer Models

Share this