Transfer Learning Technique for Accelerating Learning of Reinforcement Learning-Based Horizontal Pod Autoscaling Policy


KIPS Transactions on Computer and Communication Systems, Vol. 11, No. 4, pp. 105-112, Apr. 2022
https://doi.org/10.3745/KTCCS.2022.11.4.105,   PDF Download:  
Keywords: Kubernetes, Reinforcement Learning, Autoscaling
Abstract

Recently, many studies using reinforcement learning-based autoscaling have been performed to make autoscaling policies that are adaptive to changes in the environment and meet specific purposes. However, training the reinforcement learning-based Horizontal Pod Autoscaler(HPA) policy in a real environment requires a lot of money and time. And it is not practical to retrain the reinforcement learning-based HPA policy from scratch every time in a real environment. In this paper, we implement a reinforcement learning-based HPA in Kubernetes, and propose a transfer leanring technique using a queuing model-based simulation to accelerate the training of a reinforcement learning-based HPA policy. Pre-training using simulation enabled training the policy through simulation experience without consuming time and resources in the real environment, and by using the transfer learning technique, the cost was reduced by about 42.6% compared to the case without transfer learning technique.


Statistics
Show / Hide Statistics

Statistics (Cumulative Counts from September 1st, 2017)
Multiple requests among the same browser session are counted as one view.
If you mouse over a chart, the values of data points will be shown.


Cite this article
[IEEE Style]
Y. Jang, H. Yu, S. Kim, "Transfer Learning Technique for Accelerating Learning of Reinforcement Learning-Based Horizontal Pod Autoscaling Policy," KIPS Transactions on Computer and Communication Systems, vol. 11, no. 4, pp. 105-112, 2022. DOI: https://doi.org/10.3745/KTCCS.2022.11.4.105.

[ACM Style]
Yonghyeon Jang, Heonchang Yu, and SungSuk Kim. 2022. Transfer Learning Technique for Accelerating Learning of Reinforcement Learning-Based Horizontal Pod Autoscaling Policy. KIPS Transactions on Computer and Communication Systems, 11, 4, (2022), 105-112. DOI: https://doi.org/10.3745/KTCCS.2022.11.4.105.