PROVIDER - Modern YAML-driven load testing

Artillery

Artillery describes load scenarios in 1 YAML file covering HTTP, WebSocket, Socket.io and Playwright across 4 target types, with threshold assertions for CI gates and Lambda scale-out; the gaps are limited complex logic, paid advanced reporting and a smaller plugin ecosystem.

Developer Tools United States 2026.11.19
4pts

Detailed Notes

Pros 5 items

Describes load scenarios in 1 YAML file with multi-phase curves and no code required
Targets 4 test types: HTTP, WebSocket, Socket.io and Playwright browser flows
Builds in assertions and thresholds that act directly as 1 CI performance gate
Scales load horizontally up to 10 times through a Serverless mode on AWS Lambda
Reports latency distribution, throughput and error rates in 3+ output formats

Cons 4 items

Complex orchestration is weaker than code-based tools, with limited branching
Advanced reporting and team collaboration require a commercial subscription
The plugin ecosystem is smaller than JMeter or k6
High concurrency needs cloud execution since a single local host is limited

Service Overview

Artillery is a load testing tool for modern applications, implemented in Node.js. Scenarios are written in YAML that describes traffic curves through phases, and threshold assertions decide whether a run passes.

In the development tools category it takes the configuration-driven route: lighter than JMeter and less code-heavy than Locust, which suits teams adding performance gates to CI quickly.

Key Strengths

  • YAML driven: 1 configuration file expresses a full load curve
  • Broad scenarios: HTTP, WebSocket, Socket.io and browser flows
  • Gate friendly: built-in thresholds decide pass or fail
  • Horizontal scale: AWS Lambda amplifies generated load
  • Clear reports: 3+ output formats for pipelines

Product Ecosystem

Platform capabilities

Scenario definition, multi-phase load, variables and data files, threshold assertions, plugin extensions, report output and cloud execution.

Integrations

Works with GitHub Actions, GitLab CI, Playwright, Grafana and AWS Lambda.

Typical scenarios

API load tests, real-time communication testing, end-to-end browser performance and CI performance gates.

Limitations

  • Logic limits: complex branching is hard to express in one YAML file
  • Commercial limits: advanced reporting and collaboration are paid
  • Smaller ecosystem: fewer plugins than JMeter or k6
  • Concurrency needs cloud: single-host capacity is limited

Use Cases

  • CI performance gates (★★★★★): thresholds work out of the box
  • Real-time communication load (★★★★☆): WebSocket support is solid
  • Browser-side performance (★★★★☆): Playwright integration is rare and useful
  • Complex orchestration (★★★☆☆): code-based tools are more flexible

Pricing

Item Price Notes
Artillery open source $0 MPL-2.0 open source, commercial use allowed
Artillery Cloud Usage subscription Hosted reports and team collaboration
Cloud load costs Provider billing Lambda and containers bill by usage

FAQ

Should I choose Artillery or k6?

Both are configuration driven; Artillery supports WebSocket and browser scenarios more directly, while k6 is stronger in scripting and ecosystem.

How do I set a performance gate?

Declare ensure thresholds such as error rate and latency ceilings so the command exits non-zero when they fail.

Can I run large tests locally?

Local capacity is limited; combine --platform with cloud execution to scale traffic.

Can scenarios be reused for functional tests?

Partially, especially Playwright browser flows.

Code Examples

# load-test.yml
config:
  target: "https://api.example.com"
  phases:
    - duration: 60
      arrivalRate: 5
      name: "warm up"
    - duration: 120
      arrivalRate: 20
      name: "sustained load"
  ensure:
    thresholds:
      - http.response_time.p95: 300
      - http.codes.500: 0

scenarios:
  - name: "browse and create"
    flow:
      - get:
          url: "/items"
      - post:
          url: "/items"
          json:
            name: "demo"
# Run locally and fail when thresholds are missed
npx artillery run load-test.yml

Community & Resources

Visit Artillery Website → https://www.artillery.io/
Visit Website ↗