Reinforcement Learning approaches for Artificial Pancreas Control

Abstract

openPeople with type 1 diabetes are affected by a chronic deficiency of insulin secretion in their body; as a consequence, insulin has to be continually self-administered to keep in check their blood glucose levels. In recent years, rapid technological advancements in continuous glucose monitoring and insulin administration systems have allowed researchers to work on automated control methods for diabetes management, commonly referred to as Artificial Pancreas. The development of control algorithms in this context is a very active research area. While traditional control approaches have been the main focus so far, Reinforcement Learning (RL) seems to offer a compelling alternative framework, which has not been thoroughly explored yet. This thesis investigates the employment of several RL approaches, based on the algorithm Sarsa lambda, on in silico patients, using the FDA accepted UVa-Padova Type 1 Diabetes simulator. The way the overall representation of the problem affects the performance of the system is discussed, underlying how each component fits into the general framework proposed and evaluating the pros and cons of each method. Particular emphasis is also placed on the interpretability of both the training process and the final policies obtained. Experimental results demonstrate that classic RL methods have the potential to be a viable future approach to achieve proper control and a good degree of personalization in glycemic regulation for diabetes management.People with type 1 diabetes are affected by a chronic deficiency of insulin secretion in their body; as a consequence, insulin has to be continually self-administered to keep in check their blood glucose levels. In recent years, rapid technological advancements in continuous glucose monitoring and insulin administration systems have allowed researchers to work on automated control methods for diabetes management, commonly referred to as Artificial Pancreas. The development of control algorithms in this context is a very active research area. While traditional control approaches have been the main focus so far, Reinforcement Learning (RL) seems to offer a compelling alternative framework, which has not been thoroughly explored yet. This thesis investigates the employment of several RL approaches, based on the algorithm Sarsa lambda, on in silico patients, using the FDA accepted UVa-Padova Type 1 Diabetes simulator. The way the overall representation of the problem affects the performance of the system is discussed, underlying how each component fits into the general framework proposed and evaluating the pros and cons of each method. Particular emphasis is also placed on the interpretability of both the training process and the final policies obtained. Experimental results demonstrate that classic RL methods have the potential to be a viable future approach to achieve proper control and a good degree of personalization in glycemic regulation for diabetes management