Start a Project

Magento 2 Multi Vendor marketplace – AWS Autoscaling

Alankrit Srivastava

 When it comes to high availability server architectures, couple of things that come in mind are giant servers with huge RAM and multi-core CPUs, Load Balancer, an Auto scaling manager, a shared file system,  Cache mechanism etc. For deploying this architecture we can either build all of its components from scratch or we can lookup to someone who can provide these features with flexibility and ease so that we can focus on our application.

These high-level server architectures can be deployed for Magento 2 Multi Vendor marketplace as well. Magento 2 being one the fastest growing e-commerce platform, faces the challenges to handle its giant clients’ base. For any vendor, store high availability and data consistency is primal concern. So we need to build our Magento store to face any considerable amount of traffic.

Best way to ensure this is to keep redundancies and ability to scale it up and down as per need. AWS Autoscaling provides features to scale your AWS resources as per pre-defined scaling policies. Also, a lot of other AWS services are needed to be integrated as well in order to set up a full fledged Auto scaling Architecture.

 

 

What is Magento 2 Multi-Vendor Marketplace

 

 

Magento 2 Multi Vendor Marketplace Extension converts your Magento 2 Store into a complete online marketplace shop. Using Magento2 Marketplace Module, the seller can manage the inventory, shipment, Seller Profile page, and the Seller Collection page and much more.The seller can add multiple products like Simple, Configurable, Virtual & Downloadable.

For more detailed information, please visit https://webkul.com/blog/magento2-multi-vendor-marketplace/ .

 

 

A Brief to AWS Auto Scaling

 

 

Amazon Web Services provides Auto Scaling that monitors your applications and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. It ensures that your auto scaling group retains predefined minimum instance counts when scaling in occurs and stops the launching of further instances during scaling out process when maximum instance limit is achieved.

 

Benefits of Auto Scaling:

 

 

 

AWS Auto Scaling with Magento 2 Multi Vendor Marketplace

 

 

In this blog, we will discuss the architecture layout for building a highly available Magento 2 Multi Vendor Marketplace store.

Standalone AWS autoscaling service would not be enough due to the need of other AWS service as well to complete architecture setup. Since scope of the architecture shouldn’t be for one time setup only, we have to devise a way so that our application faces zero or minimum downtime during an update, be it code or server dependencies. 

In addition to this, code build (or update) process should be automated to an extent that concerning developers could focus on their application, not bothering about underlying architecture.

 

Architecture Components:

 

 

Our Application code (Magento 2 Multi Vendor Marketplace files) would be managed by a Source Code Management System like GitLab or Github. A webhook will be configured that will trigger when development branch is merged with Master. We will discuss about webhook integration later in this blog.

As to ensure load balancing among various identical servers, we would be using Elastic Load Balancer to distribute incoming traffic evenly among all the instances. Here we are using Classic Load Balancer.

Since, we would be having multiple EC instances, there should be a centralised files system where we could put our media files. For ensuring this, we would be using AWS Elastic File System. Magento 2 pub/media folder will reside in this and it would be mounted on all the instances.

We would be using Auto scaling to launch and terminate instances as per scaling policies. These instances would be added under elastic load balancer by themselves and would fetch media files from EFS.

We would need to setup Cloudwatch Alarms to trigger Scaling policies. Here we are using alarms for CPU utilisation monitoring.

AWS SDK for python (BOTO) would be needed to automate the process of updating ec2 AMI, Load balancer, Autoscaling group etc whenever a webhook is triggered from GitLab.

And Ofcourse we would need an Staging server of Magento 2 for initial server setup before automating the whole build process. 

 

An illustration for proposed architecture is shown below:

 

 

 

Initial Server Setup

 

 

So we have drawn the layout of our architecture, next step would be setting it up and connecting all the components of architecture.

 

(a) GITLAB Setup:

 

 

(b) Magento 2 Staging Server Setup:

 

 

(c) Linking Magento 2 with GITLAB:

 

 

(d) Setup AWS EFS to store media Files:

 

 

(e) Setup AWS Elastic Load Balancer:

 

 

(f) Create EC2 Instance AMI and configure Launch Configuration:

 

 

(g) Setup Auto Scaling Group and Cloudwatch Alarms:

 

 

(h) Setup Python SDK scripts for EC2 AMI update:

 

 

 

EC2 AMI Update Workflow

 

 

So far, all the servers have been set and our Magento 2 server is running as a Staging server with Marketplace module. Next step is to deploy our server as a production environment.

Pull the development branch from the repository and make some changes in files.

Push the code back to the development branch in repository and create a merge request. When development branch is merged to Master branch, webhook will get triggered. This webhook in return will trigger python sdk scripts.

On triggering Webhook for a request merged, scripts will do the following jobs:-

 

 

However if there’s is any requirements to install server dependencies and other packages, login to existing Magent 2  server, do the needful and create AMI of the instance.

Then updates AMI id in the file or database to be fetched next time the webhook is triggered.

The complete flowchart for the whole workflow:

 

 

 

 

 

Testing Auto Scaling and Elastic Load Balancing

 

 

To check if auto scaling is working or not, we can use siege tool for benchmarking. Siege tool will generate huge amount of requests within a short period of time which in turn will raise CPU load average.

 

 

Then as CPU load average will go beyond 5, it triggers high CPU Cloudwatch alarm which in turn notify autoscaling group to launch an EC2 instance and add it in load balancer.

As soon as load average will come below 5, it triggers Low CPU cloudwatch alarm, and autoscaling group will terminate one instance and will retain Auto scaling group size.

Try configuring Cloudwatch alarms as for request counts, bandwidth, instance counts etc and benchmark with siege and other tools as well to ensure your application has high availability. Also please ensure that theload balancer is maintaining redundancy, distributing the traffic among instances under the load balancer.

 

 

Exit mobile version