FScaler: Automatic Resource Scaling of Containers in Fog Clusters Using Reinforcement Learning

• Post by: Hani Sami
• 25 June 2021
• Comments off

Abstract

Several studies leverage fog computing as a solution to overcome cloud delays, including computation, network, and data storage. Along with the increase in demands for computing resources in fog infrastructures, heterogeneous fog devices are used towards forming highly available clusters. Existing approaches support the use of heterogeneous fogs and enable dynamic updates and management of services through containerization and orchestration technologies. However, none of the existing works proposed a proactive solution to horizontally scale these resources based on the IoT workload fluctuations, in addition to deciding on proper placement of the scaled instances on fogs with minimal cost on the fly. An effective scaling results in improving the response time and avoid service instability on fog devices. Therefore, we propose in this work FScaler, a reinforcement learning agent that horizontally scales container’s instances after studying user’s demands, and schedules the placement of newly created instances based on defined cost functions after studying the change in resources availability. The environment of FScaler is modeled as an MDP to be solved by any RL algorithm. For this work, we study the efficiency of our MDP formulation by solving the problem using SARSA. Promising results are shown through testing using a real-life dataset presenting the variation of user’s demands of a particular service and the change in resource availability over time.

Authors’ Notes

Reinforcement Learning (RL), a category of machine learning, utilizes intelligent agents to build knowledge, take actions, and adapt to the changes in its environment through a load balance between exploration and exploitation. The agent aims to maximize a certain reward or minimize a given cost. This knowledge can either be built from scratch and is called model-free or uses a model of the environment to learn faster, which is called model-based. The agent environment is formulated as a Markov Decision Process (MDP) to be solved using RL algorithms. The model-free approach can be used to learn from the environment from scratch because it is hard to model the randomly changing service demands over time. RL is proven to excel in the areas of robotics, studying behaviors, scheduling, and many more. Motivated by the advancement in RL, we propose in this paper Fscaler (Fog scaler), a model-free RL to solve our scaling problem taking into account the aforementioned challenges. Fscaler generates four combined decisions serving the horizontal resource scaling of containers in fog clusters based on defined cost functions. These four decisions are: (1) the number of containers to add, (2) the number of containers to remove, (3) an efficient placement of the service instances on fogs considering contradicting objectives and available resources, and (4) selection of fogs to stop and remove running instances that are not useful or can block other applications. Through a series of experiments, we obtain promising results showing the correctness and efficiency of our proposed MDP formulation after solving it using a model-free RL algorithm called SARSA. To the best of our knowledge, we are the first to propose the horizontal scaling of containers in fog clusters that is capable of making such decisions.

The contributions of this paper are:

• A novel architecture to account for dynamic resource scaling of containers through integrating FScaler, an RL agent, in Kubernetes fog clusters.
• A novel MDP formulation of the scaling problem, which can be later solved by various RL agents depending on the fog cluster and application scale.
• Proposing the use of SARSA to build the FScaler agent, which is proven to reach optimal policy through a series of realistic experiments.