Bond-DQN: Deep Q-Learning of Lumped-Element Systems Design via Bond Graphs

Abstract Lumped-element schematics (e.g. free-body diagrams, electrical circuit schematics) have been utilized for design synthesis in a wide variety of domains. They are invaluable for generalizing design synthesis in a process which consists of taking a set of available components and exploring the space of possible connectivity. The connectivity of a given lumped-element model (e.g. the netlist of an electrical schematic) represents a parameterized model which may be optimized using generalizable state space techniques and has the added benefit of being highly interpretable. Some recent machine learning models, particularly in the analog electrical circuit synthesis community, have attempted to leverage such lumped-element models in design synthesis or forward modeling tasks. They either learn faster neural network proxies for the forward dynamics of human-crafted lumped-element simulators or use hand crafted lumped-element representations in the networks themselves. An issue with these approaches is the models are difficult to generalize to different domains due to their reliance on a domain-specific dataset or simulator for design performance evaluation. In this work, we propose Bond-DQN, a Deep Q-Learning method to sequentially generate lumped-parameter designs for state-space based objectives. Specifically, our method uses a grammar based on bond graphs, which are an energy-based graph representation which have been historically used to simulate lumped-parameter models in a variety of domains. The combination of bond graphs with DQL allows us to accurately simulate and sample the large design space of multi-domain systems without retraining on external simulators or new datasets. We demonstrate Bond-DQN’s capability for generalizeable lumped-parameter design by benchmarking against a suspension optimization problem in which the previous SOTA utilizes the Harris Hawks Optimization algorithm, demonstrating a reduction in the peak vehicle translational and pitch acceleration of 20.5% and 4.1% respectively. Additionally, we showcase its ability for multi-domain design by applying to an electrical circuit voltage regulation problem.