I am working on fusing classical control with reinforcement-learning based control in drones.
The good thing about RL is that it is adaptive and proactive - it tunes control from data as it streams and it learns to act with delayed feedback in mind.
The bad thing is that it can be sample inefficient. It needs lots and lots of data. That is not good in a time-critical case such as when faults occur. I've come to enjoy pushing back on completely black-box control of systems, foregoing the advances of decades of theory. That helps me scale back complexity of the system if only a little domain knowledge is sprinkled on the solution.
So, first, I started using transfer learning to re-use existing policies from similar situations. However "similar" is a very open problem in the domain. I then thought that we can exploit insights we get from classical control of linear systems into these more complex domains. If, for example, we can make some simplifying assumptions (locally linear, there's some global optimum etc.). So far I am getting promising results. But there is always some data wrangling involved. I am hoping to use that in my dissertation later this year. (If anyone's hiring, feel free to reach out).
The good thing about RL is that it is adaptive and proactive - it tunes control from data as it streams and it learns to act with delayed feedback in mind.
The bad thing is that it can be sample inefficient. It needs lots and lots of data. That is not good in a time-critical case such as when faults occur. I've come to enjoy pushing back on completely black-box control of systems, foregoing the advances of decades of theory. That helps me scale back complexity of the system if only a little domain knowledge is sprinkled on the solution.
So, first, I started using transfer learning to re-use existing policies from similar situations. However "similar" is a very open problem in the domain. I then thought that we can exploit insights we get from classical control of linear systems into these more complex domains. If, for example, we can make some simplifying assumptions (locally linear, there's some global optimum etc.). So far I am getting promising results. But there is always some data wrangling involved. I am hoping to use that in my dissertation later this year. (If anyone's hiring, feel free to reach out).