Testing Models with Locust
Before testing the model with Locust, you will need to make a test deployment via MLflow. To do this, you will need to run the following command:
$ neuro-flow run deploy_inference_platform --param run_id <run-id>
You can check the required run ID in MLflow:
By running the
deploy_inference_platformjob, you gain access to the target model's binary and deploy it to the Neu.ro cluster.
Once the job is running, it will also generate a predictable URI which you can later use to access the model in Locust.
You can specify how the Neu.ro platform will use Locust via the
.neuro/live.ymlfile in the project's root folder.
jobssection of the
live.ymlfile, you will find a default
locustjob description that looks like this:
name: $[[ flow.title ]]-locust
https://demo-oss-dogs-test-inference--<user-name>.jobs.<cluster-name>.org.neu.ro/api/v1.0/predictions - if model deployed as platform job
http://seldon.<cluster-name>.org.neu.ro/seldon/seldon/<model-name>-<model-stage>/api/v1.0/predictions -if model is deployed in Seldon
- $[[ upload(volumes.src).ref_ro ]]
- $[[ upload(volumes.config).ref_ro ]]
- $[[ volumes.remote_dataset.ref_ro ]]
DOG_IDS: "n02085936, n02088094"
IMGS_DIR: $[[ volumes.remote_dataset.mount ]]/images/Images/
PYTHONPATH: $[[ volumes.src.mount ]]/..
-f $[[ volumes.src.mount ]]/locust.py --web-port 8080 -H $[[ params.endpoint_url ]]
You can always fine-tune the job depending on your needs by changing the values of various parameters such as
life_spanin the description.
To run a
locustjob, execute the following:
$ neuro-flow run locust
This job will use a
locust.pyfile specified in the
cmdsection of the job definition. This file will tell Locust what specific tests to run and how to run them. You can learn more about writing Locust files here.
Once the Locust job is up and running, you will need to open Locust in your browser by using the predictable URI generated in the previous steps.
Next, enter the locust URL of the job on the platform in the Host field and click Start swarming:
When this is done, you can modify the amount of simulated users and the spawn rate and monitor the testing process through various charts: