TY  - JOUR
T1  - Transfer Learning Technique for Accelerating Learning of Reinforcement
Learning-Based Horizontal Pod Autoscaling Policy
AU  - Jang, Yonghyeon 
AU  - Yu, Heonchang 
AU  - Kim, SungSuk 
JO  - KIPS Transactions on Computer and Communication Systems
PY  - 2022
DA  - 2022/1/30
DO  - https://doi.org/10.3745/KTCCS.2022.11.4.105
KW  - Kubernetes
KW  - Reinforcement Learning
KW  - Autoscaling
AB  - Recently, many studies using reinforcement learning-based autoscaling have been performed to make autoscaling policies that are
adaptive to changes in the environment and meet specific purposes. However, training the reinforcement learning-based Horizontal Pod
Autoscaler(HPA) policy in a real environment requires a lot of money and time. And it is not practical to retrain the reinforcement
learning-based HPA policy from scratch every time in a real environment. In this paper, we implement a reinforcement learning-based
HPA in Kubernetes, and propose a transfer leanring technique using a queuing model-based simulation to accelerate the training of a
reinforcement learning-based HPA policy. Pre-training using simulation enabled training the policy through simulation experience without
consuming time and resources in the real environment, and by using the transfer learning technique, the cost was reduced by about
42.6% compared to the case without transfer learning technique.