'Cloud Server Auto Scaling Configuration Guide: Elastically Handling Traffic

Cloud Server Auto Scaling Configuration Guide: Elastically Handling Traffic Fluctuations

Anyone running e-commerce knows this pain: two servers are plenty on a normal day, then traffic spikes 10x during a 618 or Double 11 sale — you're up at 2am manually adding machines while the site is already timing out; and after the event you forget to scale down and burn a month of wasted spend. Auto Scaling exists precisely to solve this — letting instance count rise and fall with real load instead of relying on someone watching dashboards all night. It's one of the core advantages of cloud computing over traditional servers.

Here's the working loop:

Traffic Increase → Trigger Scale-Out → Add Instances → Load Drops
Traffic Decrease → Trigger Scale-In → Remove Instances → Cost Drops

Core concepts

AWS Auto Scaling configuration

Step 1: Create a launch template. Fix the configuration new instances will use, including the User Data that runs on boot (e.g. start nginx):

# Launch template configuration
AMI: ubuntu-22.04-lts
Instance Type: t3.medium
Security Group: web-sg
Key Pair: my-key
User Data:
  #!/bin/bash
  systemctl start nginx
  systemctl enable nginx

Step 2: Create the auto scaling group. Set minimum, maximum, and desired capacity, and spread it across multiple Availability Zones:

aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --launch-template LaunchTemplateName=web-template \
  --min-size 2 --max-size 10 --desired-capacity 2 \
  --vpc-zone-identifier subnet-aaa,subnet-bbb

Step 3: Configure policies and alarms. The easiest route is Target Tracking — you only tell it "keep CPU at 50%" and the system computes scaling automatically. Or use manual adjustment backed by CloudWatch alarms:

# Scale-out policy based on CloudWatch alarm
aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web-asg \
  --policy-name cpu-scale-up \
  --scaling-adjustment 1 --adjustment-type ChangeInCapacity --cooldown 300

# The matching CloudWatch alarm
aws cloudwatch put-metric-alarm --alarm-name cpu-high \
  --metric-name CPUUtilization --namespace AWS/EC2 \
  --threshold 70 --period 120 --statistic Average

Reference: AWS Auto Scaling docs https://docs.aws.amazon.com/autoscaling/ec2/userguide/what-is-amazon-ec2-auto-scaling.html

Alibaba Cloud Elastic Scaling

The path on Alibaba Cloud: create a scaling group → configure scaling configuration (image + spec) → add scaling rules → bind scheduled or alarm tasks. A typical rule set:

Rule Name: cpu-scale-out
Trigger: CPU utilization > 75% for 5 consecutive minutes
Action: Add 1 instance
Cooldown: 300 seconds

Rule Name: cpu-scale-in
Trigger: CPU utilization < 30% for 10 consecutive minutes
Action: Remove 1 instance
Cooldown: 600 seconds

Notice the scale-in threshold is lower and its evaluation window longer than scale-out (10 minutes vs 5 minutes in the example). The reason is simple: respond fast when traffic rises, but observe longer before shrinking, to avoid the "shrink on a blip, then bounce back" oscillation.

Reference: Alibaba Cloud Elastic Scaling docs https://help.aliyun.com/zh/ess/

Scaling in is not the same as deleting machines

Scaling out is easy; scaling in is hard. When shrinking, the system picks instances to remove — without protection, an instance still serving requests may be destroyed outright:

• Enable scale-in protection so key instances running tasks are not removed automatically;
• Make the app support graceful shutdown: on SIGTERM stop accepting new requests, drain existing connections, then exit;
• After attaching the group to a load balancer, new instances go through a health-check window — don't skip "warm-up before serving traffic".

A real case: scheduled scaling before a sale

A content site with 500k daily actives combined the two approaches: on sale day, scheduled scaling grew the fleet from 4 to 20 instances before 10am (traffic ramps up overnight — dynamic policies simply can't react in time); day-to-day, a target-tracking policy held CPU around 50%. After the event, another scheduled job shrank the fleet back to 4. Result: 99.99% of requests returned within 200ms during the sale, and the bill only grew by those few active hours.

The lesson is to govern predictable traffic and unpredictable load separately — scheduled tasks handle the former, dynamic policies catch the latter, rather than expecting one policy to do both.

Best practices

Frequently asked questions

Why won't it scale out even at 100% CPU? Check whether the cooldown is too long, the alarm's evaluation window is misconfigured, or the group's max size is already reached.

Can dynamic policies handle sudden spikes? Not fast enough — CPU is a lagging indicator. For predictable events (sales, flash deals), add scheduled scaling; for unpredictable bursts, consider reserved capacity or a higher starting instance count.

What about lost sessions after scale-in? Externalize sessions and caches (Redis, database) and keep the app stateless, so instances can come and go freely.

Auto scaling is not "set it and forget it". Maintain it together with monitoring, alarms, and load-test drills. Configured right, it's a cost-saving, load-absorbing weapon; configured wrong, it's the source of 3am alarms.