VLSI Physical Design Optimization Using Deep Reinforcement Learning

Abstract

As device scaling reaches its limits with the end of Moore’s law, design costs have increased exponentially to enhance the performance of very large-scale integrated circuits (VLSI). Moreover, the increasing design complexity has diminished design productivity. To address these challenges, the roles of design technology co-optimization (DTCO) and electronic design automation (EDA) have become critical, especially in advanced technology nodes. The chip design process, encompassing DTCO and EDA, remains complex and time-consuming. Recently, integrating artificial intelligence (AI) into chip design aims to improve design efficiency by predicting design outcomes or effectively exploring the design space. AI techniques promise to reduce design iterations, time, and manpower while improving design quality. Reinforcement learning, in particular, offers significant advantages for optimizing the VLSI physical design process, which is non-deterministic and complex. Reinforcement learning provides data efficiency in chip design, where it is difficult to get sufficient design data. Moreover, reinforcement learning can achieve the improved design quality that was unseen before by maximizing reward with unlabeled data. This dissertation explores the application of reinforcement learning in optimizing various stages of VLSI chip design, from DTCO to physical design phases. In this dissertation, there are three research objectives. The first objective is to optimize chip power consumption and timing performance by exploring design and technology parameter space for back-end-of-line (BEOL) using reinforcement learning and transfer learning. The second objective is to optimize the placement outcomes by prioritizing mixed-height standard cells using reinforcement learning in the cell legalization phase. The last objective is to achieve timing closure by optimizing the routing priority of nets using reinforcement learning and transfer learning in the detailed routing and post-route optimization phases. Through the effective application of reinforcement learning, this dissertation aims to address sequential decision-making problems and high-dimensional parameter optimization in chip design, ultimately improving design quality and chip productivity. These novel approaches have the potential to revolutionize chip design by overcoming the limitations of current chip design and lowering the entry barrier to chip design, thereby increasing the efficiency of time, cost, and manpower in the semiconductor industry.