marcel blattner | May, 2025
A Conceptual Framework for an Adaptive PID Control System Utilizing Neural Network Ensembles
Abstract
Proportional-Integral-Derivative (PID) controllers are foundational in industrial automation but exhibit limitations when faced with non-linear or time-varying system dynamics due to their fixed gain parameters. This document outlines a conceptual framework for an adaptive PID control system. The proposed architecture employs an ensemble of PID controllers whose gain parameters are learnable. Furthermore, the internal setpoints for these PID controllers are dynamically modulated by two interconnected neural networks: one generating initial static setpoints based on global system state and a primary reference, and another refining these setpoints through an internal iterative process. The entire system, including PID gains and neural network parameters, is designed to be trainable end-to-end via differentiable simulation and gradient-based optimization, aiming to enhance control performance and adaptability.
Code: https://github.com/marcelbtec/DynPID
Introduction
The Proportional-Integral-Derivative (PID) controller is a ubiquitous feedback control mechanism. Its operation is defined by the classical control law, which in its continuous form can be expressed as:
$$u(t) = K_p e(t) + K_i \int_0^t e(\tau)d\tau + K_d \frac{de(t)}{dt}$$
where $u(t)$ is the control output, $e(t)$ is the error (difference between setpoint and process variable), and $K_p, K_i, K_d$ are the proportional, integral, and derivative gains, respectively. While effective for linear time-invariant systems, the performance of PID controllers can degrade significantly when applied to systems with complex non-linearities or time-varying dynamics, as their gains are typically tuned for specific operating conditions and remain fixed. This necessitates adaptive control strategies. This experiment presents a conceptual framework for an adaptive PID control system that integrates neural networks to dynamically adjust PID parameters and internal operational setpoints.
Proposed Architecture
The proposed control system architecture is structured around an ensemble of PID controllers whose behavior is coordinated and adapted by neural network components. All learnable parameters within this architecture, including PID gains and neural network weights, are denoted collectively by $\Theta$.
PID Ensemble
The system utilizes a predetermined number, $N_c$, of individual PID controllers, denoted $C_i$ for $i = 1, \ldots, N_c$. Each controller $C_i$ adheres to a discrete PID formulation. The learnable gains for each controller $C_i$ are $(K_{p,i}, K_{i,i}, K_{d,i})$, and these form a subset of the global parameter set $\Theta$.
Hierarchical Setpoint Generation and Internal Adaptation
The determination of the operational setpoint for each PID controller $C_i$ is a two-stage process, involving both static generation and dynamic refinement within an internal iterative loop. Let $r_{ext}(t)$ be the external reference signal for the overall system at time $t$, and $x(t)$ be the measured state of the plant being controlled.
Static Setpoint Network
A neural network, termed the Static Setpoint Network ($f_{SN}$), with parameters $\theta_{SN} \subset \Theta$, maps the current plant state $x(t)$ and the external reference $r_{ext}(t)$ to a vector of $N_c$ static internal setpoints $S_{static}(t)$. This can be represented as:
$$S_{static}(t) = [s_{static,1}(t), \ldots, s_{static,N_c}(t)]^T$$
$$S_{static}(t) = f_{SN}([x(t), r_{ext}(t)]; \theta_{SN})$$
These static setpoints provide initial, differentiated targets for each PID controller in the ensemble.
Dynamic Setpoint Network and Internal Iteration Loop
For each controller $C_i$, an internal iterative process, executed for $J$ discrete steps indexed by $j=0, \ldots, J-1$, refines its operational setpoint. Let $m_{i}^{(j)}(t)$ denote the output of $C_i$ at internal iteration $j$ and external time $t$. The initial output for this internal loop, $m_{i}^{(-1)}(t)$, is typically initialized to zero or a state-dependent value. The dynamic setpoint for $C_i$ at internal step $j$ is $s_{dyn,i}^{(j)}(t)$. This is initialized as $s_{dyn,i}^{(0)}(t) = s_{static,i}(t)$.
Within each internal iteration $j$ for controller $C_i$:
The error $e_{i}^{(j)}(t)$ is calculated:
$$e_{i}^{(j)}(t) = s_{dyn,i}^{(j)}(t) - m_{i}^{(j-1)}(t)$$The integral term $I_{i}^{(j)}(t)$ is updated:$$I_{i}^{(j)}(t) = I_{i}^{(j-1)}(t) + e_{i}^{(j)}(t) \Delta t_{int}$$
where $I_{i}^{(-1)}(t)$ is the persistent integral state of $C_i$ and $\Delta t_{int}$ is the internal time step. This integral term is typically subject to anti-windup clamping.
The derivative term $D_{i}^{(j)}(t)$ is computed:
$$D_{i}^{(j)}(t) = (e_{i}^{(j)}(t) - e_{i}^{(j-1)}(t)) / \Delta t_{int}$$
where $e_{i}^{(-1)}(t)$ is the error from the previous internal state.
The output of $C_i$ is then:
$$m_{i}^{(j)}(t) = K_{p,i} e_{i}^{(j)}(t) + K_{i,i} I_{i}^{(j)}(t) + K_{d,i} D_{i}^{(j)}(t)$$
A second neural network, the Dynamic Setpoint Network ($f_{DSN}$) with parameters $\theta_{DSN} \subset \Theta$, takes the static setpoint $s_{static,i}(t)$ and the current PID output $m_{i}^{(j)}(t)$ as inputs to propose a dynamic update $\Delta s_{dyn,i}^{(j)}(t)$:
$$\Delta s_{dyn,i}^{(j)}(t) = f_{DSN}([s_{static,i}(t), m_{i}^{(j)}(t)]; \theta_{DSN})$$The dynamic setpoint for the next internal iteration $j+1$ (or for the final output if $j=J-1$) is then updated using a weighting factor $\alpha \in [0,1]$:$$s_{dyn,i}^{(j+1)}(t) = (1 - \alpha) \Delta s_{dyn,i}^{(j)}(t) + \alpha r_{ext}(t)$$The final outputs of all PIDs after $J$ internal iterations are collected into a vector $M_{final}(t)$:$$M_{final}(t) = [m_{1}^{(J-1)}(t), \ldots, m_{N_c}^{(J-1)}(t)]^T$$
The states of each PID controller (integral term and previous error for derivative calculation) are preserved across external time steps $t$.
Aggregated Control Signal
The final control signal $u(t)$ applied to the plant is derived from an aggregation of the final outputs from the PID ensemble, typically the mean:
$$u(t) = \text{agg}(M_{final}(t)) = \frac{1}{N_c} \sum_{i=1}^{N_c} m_{i}^{(J-1)}(t)$$
Parameter Optimization via Differentiable Simulation
The learnable parameters $\Theta$ of the entire control architecture are optimized through a process involving simulation of the plant and controller interaction. Let the plant dynamics be described by a discrete-time state-space model:
$$x(t+1) = P(x(t), u(t))$$The objective is to minimize a loss function $L(\Theta)$ that quantifies the deviation of the plant's state $x(t)$ from the external reference $r_{ext}(t)$ over a simulation horizon $T_{sim}$:$$L(\Theta) = \sum_{k=0}^{T_{sim}-1} \mathcal{L}(x(k), r_{ext}(k))$$
A common choice for $\mathcal{L}$ is the squared error, $\mathcal{L}(x,r) = (x-r)^2$.
The parameters $\Theta$ are updated using gradient-based optimization methods, such as Adam or SGD:
$$\Theta_{new} = \Theta_{old} - \eta \nabla_{\Theta} L(\Theta_{old})$$
where $\eta$ is the learning rate. This requires that all components of the controller and the plant simulation (if part of the learning loop, e.g., for model-based reinforcement learning) be differentiable with respect to $\Theta$. The gradient computation encompasses the neural networks, the PID control laws (including their dependency on learnable gains), and the internal iterative adaptation loop.
System Characteristics and Potential Utility
The proposed framework is designed to exhibit several key characteristics. Primarily, it aims to provide enhanced adaptive capabilities, allowing the control system to adjust its behavior in response to variations in plant dynamics or environmental disturbances. This adaptation stems from both the learnable PID gains and the dynamic adjustment of internal setpoints by the neural network components. Consequently, it is hypothesized that such a system may achieve superior reference tracking and disturbance rejection performance compared to fixed-gain PID controllers, particularly in complex or non-linear scenarios. The neural networks, through the learning process, can approximate complex, non-linear control mappings that would be challenging to design manually. A potential practical benefit is the reduction of manual tuning effort, as the system learns its operational parameters from data or simulation, guided by the specified loss function.
Considerations and Future Work
Several considerations are pertinent to the practical realization and deployment of this framework. The efficacy of simulation-based training is highly dependent on the fidelity of the plant model $P(x,u)$. Discrepancies between the model and the real plant (the sim-to-real gap) can impact performance. The computational demand, particularly from the internal iteration loop involving multiple PID evaluations and neural network inferences per control cycle, may be substantial and needs to be assessed for specific applications. While the internal dynamics can be recorded and analyzed, the interpretability of the control policies embedded within the neural network weights may be less direct than for classical controllers. Future work could explore methods for robust sim-to-real transfer, investigate the trade-offs between the number of internal iterations $J$ and performance, and develop techniques for enhancing the interpretability of the learned components. Hybrid approaches combining model-free reinforcement learning elements could also be considered to reduce model dependency.
Wrap Up
The conceptual framework presented describes an adaptive PID control system that integrates an ensemble of PID controllers with learnable gains and neural networks for dynamic setpoint generation and adaptation. By leveraging end-to-end differentiability and gradient-based optimization, the system has the potential to learn complex control strategies and adapt to varying operating conditions. While further research and empirical validation are necessary, this architecture offers a promising direction for advancing the capabilities of PID-based control systems in challenging applications.