> ## Documentation Index
> Fetch the complete documentation index at: https://vastai-80aa3a82-cln-2792-remove-disable-bundling.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Automated Performance Testing

> Learn about the performance testing process in Vast.ai Serverless.

<script
  type="application/ld+json"
  dangerouslySetInnerHTML={{
__html: JSON.stringify({
"@context": "https://schema.org",
"@type": "TechArticle",
"headline": "Automated Performance Testing in Vast.ai Serverless",
"description": "An overview of how Vast.ai Serverless uses automated benchmark testing to guide scaling, routing, and cost-performance optimization.",
"author": {
  "@type": "Organization",
  "name": "Vast.ai"
},
"articleSection": "Serverless Documentation",
"keywords": ["performance testing", "benchmarking", "serverless", "scaling", "pyworker", "vast.ai"]
})
}}
/>

Vast Serverless relies on **benchmark testing** to determine the most cost-effective GPU when scaling up (which workers to recruit), routing requests (which workers have available capacity), and scaling down (which workers to release).

This benchmark is part of the **PyWorker configuration** within the SDK and is an integral component of how Vast Serverless operates.

## How Benchmark Testing Works

When a new Workergroup is created, the serverless engine enters a **learning phase**. During this phase, it recruits a variety of machine types from those specified in `search_params`. Each new worker runs the user-configured benchmark and evaluates performance, which are reported to the serverless engine.

As traffic scales up and down, the serverless engine builds an **application-specific understanding of cost vs. performance**, which it then uses to make informed decisions about future worker recruitment and release.

## Best Practices for Initial Scaling

The speed at which the serverless engine “settles” into the most cost-effective mix of workers can vary depending on how quickly workers are recruited and released. Because of this, it is recommended to apply a **test load during the first day of operation** to help the system efficiently explore and converge on optimal hardware choices.

Best practice is to scale to double the number of expected required workers, then scale back down, 3 separate times.

## Simulating Load

For examples of how to simulate load against your endpoint, see the client examples in the Vast SDK repository:

[https://github.com/vast-ai/vast-sdk/blob/main/examples/client/vllm\_load\_example.py](https://github.com/vast-ai/vast-sdk/blob/main/examples/client/vllm_load_example.py)