WEKA installation on AWS
This section provides detailed instructions on installing a WEKA system on AWS.
Last updated
If you already have an AWS account and are familiar with AWS's basic concepts and services, you can skip this section.
To install a WEKA system in AWS, you need to create an AWS account.
Make sure you are familiar with the following concepts and services that are used as part of the WEKA system deployment:
IAM - Identity and access management
VPCs, subnets, and security groups
S3 - Object store (to be used for tiering data)
During the deployment of the WEKA system, the EC2 instances require access to the internet to download the WEKA software. For this reason, you need to deploy the WEKA system in one of the following deployment types in AWS:
Public subnet: Use a public subnet within your VPC with an internet gateway, and allow public IP addresses for your instances.
Private subnet with NAT Gateway: Create a private subnet with a route to a NAT gateway with an elastic IP in the public subnet.
Private subnet using WEKA VPC endpoint: Requires the creation of a prerequisites stack (once per VPC) that creates the necessary resources.
Private subnet using custom proxy: Requires the creation of a prerequisites stack (once per VPC) that creates the necessary resources.
The following diagrams illustrate the components of the public subnet and private subnet with NAT gateway deployment types in AWS:
Update the number of vCPU limits in EC2
By default, AWS does not provide enough vCPUs to install a WEKA system. Use the Limits Calculator for your region from the AWS EC2 dashboard.
Procedure
On the AWS EC2 dashboard, select the Limits option from the left menu.
2. In the Limits Calculator, do the following:
In the Current Limit, set the number of vCPUs you currently have for a region.
In the vCPUs needed, set the required number of vCPUs for your specific deployment.
Select the Request on-demand limit increase link to get more vCPUs.
Note: vCPU increase is not an instant action and can take minutes to days for AWS to evaluate and approve your request.
The following example shows the required vCPUs for a six servers cluster with two clients of type i3en.2xlarge. This example is the smallest type of instance for a WEKA system deployment.
After the installation on AWS best practices
Backup and recovery
Resiliency
The Weka system is a distributed cluster protected from 2 or 4 failure domain failures, providing fast rebuild times as described in the Weka system overview section.
Instance failure
If an instance failure occurs, the Weka system rebuilds the data. Add a new instance to the cluster to regain the reduced compute and storage due to the instance failure.
Upload snapshots to S3
It is advisable to use periodic (incremental) snapshots to back up the data and protect it from multiple EC2 instances failures.
The recovery point objective (RPO) is determined by the cadence in which the snapshots are taken and uploaded to S3. The RPO changes between the type of data, regulations, and company policies, but it is advisable to upload at least daily snapshots (Snap-To-Object) of the critical filesystems.
If a failure occurs and it is required to recover from a backup, spin up a cluster using the Self-Service Portal or CloudFormation, and create filesystems from those snapshots. You do not need to wait for the data to reach the EC2 volumes. It is instantly accessible through S3. The recovery time objective (RTO) for this operation mainly depends on the time it takes to deploy the CloudFormation stack and is typically below 30 min.
Cross AZ failure
See Protecting Data Against AWS Availability Zone Failures.
Region failure
Using Weka snapshots uploaded to S3 combined with S3 cross-region replication enables the protection from an AWS region failure.
SSH keys rotation
For security reasons, it is advisable to rotate the SSH keys used for the EC2 instances.
To rotate the SSH keys, follow these steps:
