A Cost-Effective and Scalable Processing of Heavy Workload with AWS Batch

Abstract

Recent technological advancements in the IT field have pushed many products and technologies into the cloud. In the present scenario, the cloud service providers mainly focus on the delivery of IT services and technologies rather than throughput. In this research paper, we used a scalable cost-effective approach to configure AWS Batch with AWS Fargate and CloudFormation and implemented it in order to handle a heavy workload. The AWS service configuration procedure, GitHub repository, and Docker desktop applications have been clearly described in this work. A cost-effective configuration and architecture of AWS Batch processing are given to provide high throughput. The processing of heavy workload by AWS Batch is represented in terms of execution time and the result shows that the concurrent execution reduces the execution time. To enhance the throughput heavy workload using batch processing an "Amazone FSx for Lustre" can also be used.