autocannon

An HTTP/1.1 benchmarking tool written in node, greatly inspired by [wrk][wrk] and [wrk2][wrk2], with support for HTTP pipelining and HTTPS. On my box, autocannon can produce more load than wrk and wrk2, see limitations for more details.

• Installation
• Usage
• API
• Acknowledgements
• License

Install

npm i autocannon -g

or if you want to use the API or as a dependency:

npm i autocannon --save

Usage

Command Line

Usage: autocannon [opts] URL

URL is any valid HTTP or HTTPS URL.
If the PORT environment variable is set, the URL can be a path. In that case 'http://localhost:$PORT/path' will be used as the URL.

Available options:

  -c/--connections NUM
        The number of concurrent connections to use. default: 10.
  -p/--pipelining NUM
        The number of pipelined requests to use. default: 1.
  -d/--duration SEC
        The number of seconds to run the autocannon. default: 10.
  -a/--amount NUM
        The number of requests to make before exiting the benchmark. If set, duration is ignored.
  -L NUM
        The number of milliseconds to elapse between taking samples. This controls the sample interval, & therefore the total number of samples, which affects statistical analyses. default: 1.
  -S/--socketPath
        A path to a Unix Domain Socket or a Windows Named Pipe. A URL is still required to send the correct Host header and path.
  -w/--workers
        Number of worker threads to use to fire requests.
  -W/--warmup
       Use a warm up interval before starting sampling.
       This enables startup processes to finish and traffic to normalize before sampling begins
       use -c and -d sub args e.g. `--warmup [ -c 1 -d 3 ]`
  --on-port
        Start the command listed after -- on the command line. When it starts listening on a port,
        start sending requests to that port. A URL is still required to send requests to
        the correct path. The hostname can be omitted, `localhost` will be used by default.
        If the command after -- is `node <script>`, this flag is optional and assumed to be `true`.
  -m/--method METHOD
        The HTTP method to use. default: 'GET'.
  -t/--timeout NUM
        The number of seconds before timing out and resetting a connection. default: 10
  -T/--title TITLE
        The title to place in the results for identification.
  -b/--body BODY
        The body of the request.
        NOTE: This option needs to be used with the '-H/--headers' option in some frameworks
  -F/--form FORM
        Upload a form (multipart/form-data). The form options can be a JSON string like
        '{ "field 1": { "type": "text", "value": "a text value"}, "field 2": { "type": "file", "path": "path to the file" } }'
        or a path to a JSON file containing the form options.
        When uploading a file the default filename value can be overridden by using the corresponding option:
        '{ "field name": { "type": "file", "path": "path to the file", "options": { "filename": "myfilename" } } }'
        Passing the filepath to the form can be done by using the corresponding option:
        '{ "field name": { "type": "file", "path": "path to the file", "options": { "filepath": "/some/path/myfilename" } } }'
  -i/--input FILE
        The body of the request. See '-b/body' for more details.
  -H/--headers K=V
        The request headers.
  --har FILE
        When provided, Autocannon will use requests from the HAR file.
        CAUTION: you have to specify one or more domains using URL option: only the HAR requests to the same domains will be considered.
        NOTE: you can still add extra headers with -H/--headers but -m/--method, -F/--form, -i/--input -b/--body will be ignored.
  -B/--bailout NUM
        The number of failures before initiating a bailout.
  -M/--maxConnectionRequests NUM
        The max number of requests to make per connection to the server.
  -O/--maxOverallRequests NUM
        The max number of requests to make overall to the server.
  -r/--connectionRate NUM
        The max number of requests to make per second from an individual connection.
  -R/--overallRate NUM
        The max number of requests to make per second from all connections.
        connection rate will take precedence if both are set.
        NOTE: if using rate limiting and a very large rate is entered which cannot be met, Autocannon will do as many requests as possible per second.
        Also, latency data will be corrected to compensate for the effects of the coordinated omission issue.
        If you are not familiar with the coordinated omission issue, you should probably read [this article](http://highscalability.com/blog/2015/10/5/your-load-generator-is-probably-lying-to-you-take-the-red-pi.html) or watch this [Gil Tene's talk](https://www.youtube.com/watch?v=lJ8ydIuPFeU) on the topic.
  -C/--ignoreCoordinatedOmission
        Ignore the coordinated omission issue when requests should be sent at a fixed rate using 'connectionRate' or 'overallRate'.
        NOTE: it is not recommended to enable this option.
        When the request rate cannot be met because the server is too slow, many request latencies might be missing and Autocannon might report a misleading latency distribution.
  -D/--reconnectRate NUM
        The number of requests to make before resetting a connections connection to the
        server.
  -n/--no-progress
        Don't render the progress bar. default: false.
  -l/--latency
        Print all the latency data. default: false.
  -I/--idReplacement
        Enable replacement of `[<id>]` with a randomly generated ID within the request body. e.g. `/items/[<id>]`. default: false.
  -j/--json
        Print the output as newline delimited JSON. This will cause the progress bar and results not to be rendered. default: false.
  -f/--forever
        Run the benchmark forever. Efficiently restarts the benchmark on completion. default: false.
  -s/--servername
        Server name for the SNI (Server Name Indication) TLS extension. Defaults to the hostname of the URL when it is not an IP address.
  -x/--excludeErrorStats
        Exclude error statistics (non-2xx HTTP responses) from the final latency and bytes per second averages. default: false.
  -E/--expectBody EXPECTED
        Ensure the body matches this value. If enabled, mismatches count towards bailout.
        Enabling this option will slow down the load testing.
  --renderStatusCodes
        Print status codes and their respective statistics.
  --cert
        Path to cert chain in pem format
  --key
        Path to private key for specified cert in pem format
  --ca
        Path to trusted ca certificates for the test. This argument accepts both a single file as well as a list of files
  --debug
        Print connection errors to stderr.
  -v/--version
        Print the version number.
  -V/--verbose
        Print the table with results. default: true.
  -h/--help
        Print this menu.

autocannon outputs data in tables like this:

Running 10s test @ http://localhost:3000
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 0 ms │ 0 ms │ 0 ms  │ 1 ms │ 0.02 ms │ 0.16 ms │ 16.45 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┬─────────┐
│ Stat      │ 1%      │ 2.5%    │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min     │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Req/Sec   │ 20623   │ 20623   │ 25583   │ 26271   │ 25131.2 │ 1540.94 │ 20615   │
├───────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┼─────────┤
│ Bytes/Sec │ 2.29 MB │ 2.29 MB │ 2.84 MB │ 2.92 MB │ 2.79 MB │ 171 kB  │ 2.29 MB │
└───────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┴─────────┘

Req/Bytes counts sampled once per second.

251k requests in 10.05s, 27.9 MB read

There are two tables: one for the request latency, and one for the request volume.

The latency table lists the request times at the 2.5% percentile, the fast outliers; at 50%, the median; at 97.5%, the slow outliers; at 99%, the very slowest outliers. Here, lower means faster.

The request volume table lists the number of requests sent and the number of bytes downloaded. These values are sampled once per second. Higher values mean more requests were processed. In the above example, 2.29 MB was downloaded in 1 second in the worst case (slowest 1%). Since we only ran for 10 seconds, there are just 10 samples, the Min value and the 1% and 2.5% percentiles are all the same sample. With longer durations these numbers will differ more.

When passing the -l flag, a third table lists all the latency percentiles recorded by autocannon:

┌────────────┬──────────────┐
│ Percentile │ Latency (ms) │
├────────────┼──────────────┤
│ 0.001      │ 0            │
├────────────┼──────────────┤
│ 0.01       │ 0            │
├────────────┼──────────────┤
│ 0.1        │ 0            │
├────────────┼──────────────┤
│ 1          │ 0            │
├────────────┼──────────────┤
│ 2.5        │ 0            │
├────────────┼──────────────┤
│ 10         │ 0            │
├────────────┼──────────────┤
│ 25         │ 0            │
├────────────┼──────────────┤
│ 50         │ 0            │
├────────────┼──────────────┤
│ 75         │ 0            │
├────────────┼──────────────┤
│ 90         │ 0            │
├────────────┼──────────────┤
│ 97.5       │ 0            │
├────────────┼──────────────┤
│ 99         │ 1            │
├────────────┼──────────────┤
│ 99.9       │ 1            │
├────────────┼──────────────┤
│ 99.99      │ 3            │
├────────────┼──────────────┤
│ 99.999     │ 15           │
└────────────┴──────────────┘

This can give some more insight if a lot (millions) of requests were sent.

Programmatically

'use strict'

const autocannon = require('autocannon')

autocannon({
  url: 'http://localhost:3000',
  connections: 10, //default
  pipelining: 1, // default
  duration: 10 // default
}, console.log)

// async/await
async function foo () {
  const result = await autocannon({
    url: 'http://localhost:3000',
    connections: 10, //default
    pipelining: 1, // default
    duration: 10 // default
  })
  console.log(result)
}

Workers

In workers mode, autocannon uses instances of Node's Worker class to execute the load tests in multiple threads.

The amount and connections parameters are divided amongst the workers. If either parameter is not integer divisible by the number of workers, the per-worker value is rounded to the lowest integer, or set to 1, whichever is the higher. All other parameters are applied per-worker as if the test were single-threaded.

NOTE: Unlike amount and connections, the "overall" parameters, maxOverallRequests and overallRate, are applied per worker. For example, if you set connections to 4, workers to 2 and maxOverallRequests to 10, each worker will receive 2 connections and a maxOverallRequests of 10, resulting in 20 requests being sent.

'use strict'

const autocannon = require('autocannon')

autocannon({
  url: 'http://localhost:3000',
  connections: 10, //default
  pipelining: 1, // default
  duration: 10, // default
  workers: 4
}, console.log)

NOTE: When in workers mode, you need to pass in an absolute file path to all the options that accept a function. This is because a function passed into the main process can not be cloned and passed to the worker. So instead, it needs a file that it can require. The options with this behaviour are shown in the below example

'use strict'

const autocannon = require('autocannon')

autocannon({
  // ...
  workers: 4,
  setupClient: '/full/path/to/setup-client.js',
  verifyBody: '/full/path/to/verify-body.js'
  requests: [
    {
      // ...
      onResponse: '/full/path/to/on-response.js'
    },
    {
      // ...
      setupRequest: '/full/path/to/setup-request.js'
    }
  ]
}, console.log)

API

autocannon(opts[, cb])

Start autocannon against the given target.

• opts: Configuration options for the autocannon instance. This can have the following attributes. REQUIRED.
  • url: The given target. Can be HTTP or HTTPS. More than one URL is allowed, but it is recommended that the number of connections is an integer multiple of the URL. REQUIRED.
  • socketPath: A path to a Unix Domain Socket or a Windows Named Pipe. A url is still required to send the correct Host header and path. OPTIONAL.
  • workers: Number of worker threads to use to fire requests.
  • connections: The number of concurrent connections. OPTIONAL default: 10.
  • duration: The number of seconds to run the autocannon. Can be a timestring. OPTIONAL default: 10.
  • amount: A Number stating the number of requests to make before ending the test. This overrides duration and takes precedence, so the test won't end until the number of requests needed to be completed is completed. OPTIONAL.
  • sampleInt: The number of milliseconds to elapse between taking samples. This controls the sample interval, & therefore the total number of samples, which affects statistical analyses. default: 1.
  • timeout: The number of seconds to wait for a response before. OPTIONAL