Control Systems

I came to control and reinforcement learning from chemical engineering, and in that passage the general formulation of the sequential decision problem was valuable to me, as a means of reading across fields in which I held no native vocabulary (thanks to legends like Dimitri Bertsekas and Warren Powell). However, for most researchers and practitioners, the object of interest is not the general formulation but the reduced problem that a particular field actually studies. ...

Automation runs safety-critical processes well inside their design envelope. Abnormal situations still fall to human operators, and their mismanagement is a leading contributor to process-safety incidents. Full autonomy in other domains usually leans on a safe stop. A chemical plant does not have that recourse because stopping is itself a hazardous and expensive operation. This project asks whether a general-purpose large reasoning model (LRM), with no task-specific training and only the information available to an operator, can manage abnormal situations at run time, provided it acts through an interface that cannot express an unsafe command. ...

In this post, I would like to think about agents that already run our world. They are not the fancy agents that have come to dominate the technology discourse. They are time-tested workhorses. To understand the true challenge of creating a trustworthy autonomous system, I propose to think about the most successful and widely deployed agent in history: the Proportional-Integral-Derivative (PID) controller. I choose the PID controller precisely because it is simple, real, and tested time again in the real world. It is a decision-making policy in its purest form. While the control algorithm itself is well-studied, the enormous amount of engineering required to make it operate reliably highlights the critical gap between an abstract algorithm and a functioning agent. Analyzing these necessary steps provides a crucial reality check for any credible claim about autonomy. ...