AI-based Resource Provisioning of IoE Services in 6G: A Deep Reinforcement Learning Approach

• Post by: Hani Sami
• 25 June 2021
• Comments off

Abstract

Currently, researchers have motivated a vision of 6G for empowering the new generation of the Internet of Everything (IoE) services that are not supported by 5G. In the context of 6G, more computing resources are required, a problem that is dealt with by Mobile Edge Computing (MEC). However, due to the dynamic change of service demands from various locations, the limitation of available computing resources of MEC, and the increase in the number and complexity of IoE services, intelligent resource provisioning for multiple applications is vital. To address this challenging issue, we propose in this paper IScaler, a novel intelligent and proactive IoE resource scaling and service placement solution. IScaler is tailored for MEC and benefits from the new advancements in Deep Reinforcement Learning (DRL). Multiple requirements are considered in the design of IScaler’s Markov Decision Process. These requirements include the prediction of the resource usage of scaled applications, the prediction of available resources by hosting servers, performing combined horizontal and vertical scaling, as well as making service placement decisions. The use of DRL to solve this problem raises several challenges that prevent the realization of IScaler’s full potential, including exploration errors and long learning time. These challenges are tackled by proposing an architecture that embeds an Intelligent Scaling and Placement module (ISP). ISP utilizes IScaler and an optimizer based on heuristics as a bootstrapper and backup. Finally, we use the Google Cluster Usage Trace dataset to perform real-life simulations and illustrate the effectiveness of IScaler’s multi-application autonomous resource provisioning.

Authors’ Notes

We propose in this paper IScaler, a DRL-based resource scaling and service placement solution combined with a suitable architecture for integration in clustering environments. In one of our previous work we proposed FScaler which offers horizontal scaling of a single application using the SARSA RL algorithm was proposed. The MDP design of IScaler is well studied to consider predicting the change in user demands reflected by the resource usage and the change of available resources on hosting nodes in the cluster. The efficient IScaler predictions allows performing proactive decisions. Moreover, the service placement solution is embedded in the state representation of the MDP and performs combined horizontal and vertical scaling in the action space. IScaler uses a custom-built model-free DRL algorithm that utilizes our designed MDP to build an optimal control policy. We also propose embedding IScaler in an Intelligent Scaling and Placement module (ISP) module that runs IScaler, an optimizer module (thereafter called Optimizer), and a solution switch module (thereafter called Solution Switch). The optimizer runs a heuristic-based solution to perform scaling when IScaler is not ready. Once IScaler learning converges, the Solution Switch is triggered to shift from the use of the Optimizer to start executing IScaler’s decision in the environment.

The contributions of this work are summarized as follows:

• A novel architecture that embeds ISP as a service for bootstrapping IScaler, our DRL-based solution.
• An MDP design for building IScaler, while respecting the MEC requirements.
• A custom DQN algorithm to build the novel IScaler.

A series of experiments using the Google Cluster Usage Trace dataset are conducted. Through these experiments, we illustrate the ability of IScaler to perform optimal auto-scaling decisions in multi-application container-based clustering environments. We also experiment with the agent behavior during the changes in demand compared to the recent Dyna-Q solution. Finally, we illustrate the advantages of using our ISP to overcome existing DRL limitations.