Comment on page
Testing Models with Locust
Locust is an open-source load testing tool. You can use it in conjunction with Neu.ro to test your models and make sure they work as expected before deploying them to production.
Before testing the model with Locust, you will need to make a test deployment via MLflow. To do this, you will need to run the following command:
$ neuro-flow run deploy_inference_platform --param run_id <run-id>
You can check the required run ID in MLflow:
By running the
deploy_inference_platform job, you gain access to the target model's binary and deploy it to the Neu.ro cluster.Once the job is running, it will also generate a predictable URI which you can later use to access the model in Locust.
You can specify how the Neu.ro platform will use Locust via the
.neuro/live.yml file in the project's root folder.In the
jobs section of the live.yml file, you will find a default locust job description that looks like this:locust:
image: locustio/locust:1.4.1
name: $[[ flow.title ]]-locust
http_port: 8080
http_auth: False
life_span: 1d
detach: True
browse: True
params:
endpoint_url:
default: ~
descr: |
Examples:
https://demo-oss-dogs-test-inference--<user-name>.jobs.<cluster-name>.org.neu.ro/api/v1.0/predictions - if model deployed as platform job
http://seldon.<cluster-name>.org.neu.ro/seldon/seldon/<model-name>-<model-stage>/api/v1.0/predictions -if model is deployed in Seldon
volumes:
- $[[ upload(volumes.src).ref_ro ]]
- $[[ upload(volumes.config).ref_ro ]]
- $[[ volumes.remote_dataset.ref_ro ]]
env:
DOG_IDS: "n02085936, n02088094"
IMGS_DIR: $[[ volumes.remote_dataset.mount ]]/images/Images/
PYTHONPATH: $[[ volumes.src.mount ]]/..
cmd: |
-f $[[ volumes.src.mount ]]/locust.py --web-port 8080 -H $[[ params.endpoint_url ]]
You can always fine-tune the job depending on your needs by changing the values of various parameters such as
http_port or life_span in the description.To run a
locust job, execute the following:$ neuro-flow run locust
This job will use a
locust.py file specified in the cmd section of the job definition. This file will tell Locust what specific tests to run and how to run them. You can learn more about writing Locust files here.Once the Locust job is up and running, you will need to open Locust in your browser by using the predictable URI generated in the previous steps.
Next, enter the locust URL of the job on the platform in the Host field and click Start swarming:
When this is done, you can modify the amount of simulated users and the spawn rate and monitor the testing process through various charts:
