Multiscale Modeling of Chemical Reactors with Neural Networks

Project Details

• Role: Research & Development
• Project Type: University Project
• Core Concept: Physics-Informed ML

Technologies Used

• Python
• PyTorch
• torchdiffeq Library

Project Overview

This project aimed to autonomously discover the kinetic parameters of chemical reactions using a novel approach: the Chemical Reaction Neural Network (CRNN). Unlike traditional black-box models, a CRNN’s architecture is designed to inherently respect the fundamental laws of physics, making its parameters (weights and biases) directly interpretable as stoichiometric coefficients and reaction constants. The primary challenge was to train this network using only concentration data, which required integrating a reactor model and an Ordinary Differential Equation (ODE) solver directly into the training loop.

The Challenge: Solving Stiff ODEs

Training the CRNN involved solving an initial value problem (IVP) at every epoch. The stability and efficiency of this process depended entirely on the chosen numerical ODE solver. Our investigation revealed significant challenges:

Explicit Solvers (e.g., Runge-Kutta 4)

While fast, these solvers proved unstable. They failed to converge when encountering “stiff” training samples—scenarios with widely varying reaction rates—causing the entire training process to collapse.

Adaptive Solvers (e.g., Dopri5)

These solvers improved stability by automatically reducing step sizes for difficult samples. However, this came at a massive computational cost, making the training time impractically long, especially during backpropagation.

Implicit Solvers (e.g., Adams-Moulton)

Implicit methods were far more robust and could handle the stiff problems. They became the foundation of our solution, but even they could be slowed down or derailed by particularly problematic samples in complex reaction networks.

Our Solution: Automatic Sample Control

The key insight was that not all training samples are equally important at every epoch. Some “critical” samples could temporarily be ignored to allow the network to learn from easier ones first. To achieve this, we developed and implemented a proof-of-concept algorithm for automatic sample control:

• Dynamic Evaluation: Periodically during training (e.g., every 5-10 epochs), the mean absolute error (MAE) is calculated for each sample.
• Critical Sample Identification: Any sample with an MAE significantly higher (e.g., >40%) than the batch average is flagged as “critical.”
• Temporary Exclusion: These critical samples are temporarily excluded from the training process for a set number of subsequent epochs.
• Continuous Process: This cycle repeats, allowing samples to be classified as critical or non-critical dynamically as the network’s weights evolve.

Results and Conclusion

The combination of a robust implicit ODE solver and our automatic sample control algorithm proved highly effective. This approach successfully enabled the training process to converge where it previously failed, even for complex and stiff reaction systems. The final trained CRNN was able to accurately determine all stoichiometric coefficients and reaction rate constants for the given reaction network. This work serves as a successful proof-of-concept for using physics-informed neural networks to automate the discovery of chemical kinetics, demonstrating that intelligently managing training data is crucial for solving complex scientific modeling problems.

---

Want to know more about the project?

Are you interested in the topic or want to read the full report?

Email Me