Optimizing Deadlock Detection Strategies for Nested Transactions in Complex Systems

Abstract Background Deadlock detection in nested transactions is a critical aspect of maintaining system stability and ensuring efficient transaction processing. In nested transaction (NT) systems, deadlocks can occur not only between top-level transactions but also among subtransactions, leading to intricate cycles that complicate deadlock resolution. Traditional approaches, which often rely on examining the entire Wait-For Graph (WFG) for cycles, can be time-consuming and resource-intensive. As the complexity of transactions increases, efficient deadlock detection methods become increasingly essential for optimizing resource utilization and overall system performance. Objective The primary objective of this research is to enhance deadlock detection techniques in nested transactions by developing an efficient algorithm based on Depth-First Search (DFS). The focus is on identifying hierarchical deadlocks that arise from subtransactions, aiming to improve the speed and accuracy of deadlock detection in complex transaction scenarios. Method We propose a novel approach that models transaction relationships as a directed graph, where nodes represent transactions and edges denote dependencies. The DFS algorithm is iteratively applied to the transaction graph to identify cycles indicative of potential deadlocks. Upon detection, the algorithm provides insights into the involved transactions, the deadlock cycle, and indirect dependencies. Additionally, we compute key performance metrics such as execution time, memory usage, system throughput, response time, and success ratio during the deadlock detection process. This approach was implemented and validated in a MATLAB simulation environment. Result The implementation of the DFS-based algorithm demonstrated significant improvements in deadlock detection efficiency within nested transactional systems. The results indicated that the algorithm effectively identifies deadlocks arising from hierarchical dependencies, providing detailed insights into transaction relationships. Key metrics showed favourable performance in terms of execution time, memory usage, and overall system throughput, highlighting the algorithm's capability to manage varying transaction complexities. Conclusion This research contributes to the advancement of deadlock detection methods in nested transactions by presenting a robust DFS-based algorithm that effectively addresses the challenges of hierarchical deadlocks. The findings underscore the importance of efficient deadlock detection techniques for optimizing resource utilization and enhancing the performance of concurrent systems. Our approach lays the groundwork for future research and development of more resilient and dependable transaction management systems.