Reinforcement Learning for Automated Flowsheet Synthesis of Steady-State Processes (2021-2024) Funded by the German Research Foundation (DFG)

Project Description

Flowsheet synthesis is a central element in the conceptual design of chemical processes. By nature this process is highly creative, and hard to formalize for automation. Prevalent methods of computer-aided flowsheet synthesis however are formalized algorithms using knowledge-bases rules for choosing between alternative planning steps, often stemming from superstructure optimization.

The aim of this research project is to use modern reinforcement learning (RL) methods for automated but creative flowsheet synthesis of steady-state chemical processes. Here, a learning agent sequentially generates process flowsheets by interacting with a process simulator, which provides feedback on the quality of the generated flowsheets based on a cost function. The agent has no prior knowledge of chemical engineering and learns exclusively through this automated interaction with the simulator. Only the process simulator contains the a priori available physical knowledge, i.e. physical-chemical material properties and a series of general apparatus models.

The central working hypothesis is that this approach is capable of independently generating feasible process flowsheets which are nearly optimal with respect to a given cost function.

The project is thus divided into two closely intertwined parts: On the one hand, robust and fast simulation environments for specific problems of interest must be developed and implemented. For this purpose, simplified shortcut and surrogate models for the apparatuses are first used (Burger Group, Chemical and Thermal Process Engineering, Technical University of Munich). On the other hand (Grimm group, Bioinformatics and Machine Learning, University of Applied Sciences Weihenstephan-Triesdorf and Technical University of Munich, Campus Straubing), new RL methods must be developed to overcome the combinatorial challenges of flowsheet synthesis: the planning horizon as well as the number of possible planning decisions (discrete and continuous) are extremely large. In addition, it is unclear a priori how to evaluate unfinished flowsheets amid such a sequential synthesis, which makes it difficult for the agent to learn.

In order to better address these challenges, the problem of flowsheet synthesis is transformed into a competitive two-player game. This leads to a generalized formulation and analysis as a combinatorial planning problem and allows the application of algorithms developed for complex highly combinatorial game applications (Chess, Go).