Comment on page

Hyperparameter Tuning with NNI


This tutorial demonstrates how to use NNI (an open-source tool from Microsoft) for Hyperparameter Tuning on You will create a new project, integrate it with NNI and run multiple tuning workers to speed up the search process.
Before moving forward with the tutorial, make sure you have CLI and cookiecutter installed.

Creating a project

To create a new project, run:
$ cookiecutter gh:neuro-inc/cookiecutter-neuro-project --checkout release
cd <project-id>

Populating the Experiment Code and Integrating With

We're going to use this NNI example code with a MNIST dataset. Put the file to the modules folder and search_space.json to the config folder.
Then, add the following lines to requirements.txt:
nni==2.0 # Required for Hyper-parameter search
neuro-sdk # Required by NNI integration script
Jinja2>=2.11.2 # Required by NNI integration script
We are now ready to build our image:
$ neuro-flow build myimage
While Docker builds our image, we can continue setting up the NNI integration.
Add the following lines at the end of .neuro/live.yml:
image: $[[ images.myimage.ref ]]
life_span: 1d
multi: true
detach: true
- $[[ volumes.code.ref_ro ]]
- $[[ volumes.config.ref_ro ]]
- $[[ volumes.project.ref ]]
bash: |
sleep infinity
image: $[[ images.myimage.ref ]]
life_span: 1d
http_port: 8080
http_auth: false
pass_config: true
detach: true
browse: true
- $[[ volumes.code.ref_ro ]]
- $[[ volumes.config.ref ]]
- $[[ volumes.project.ref ]]
bash: |
cd $[[ volumes.config.mount ]] && python3
cd $[[ volumes.project.mount ]] && USER=root nnictl create --config $[[ volumes.config.mount ]]/nni-config.yml -f
This will add some new jobs to work with NNI.
Finally, put nni-config-template.yml and to the config folder and Makefile to the root folder of your project.
Once neuro-flow build myimage completes, your project is ready for running on

Running the Tuning Jobs

The only thing left is to run
make hypertrain
This command will:
  • Run 3 worker nodes. They can be configured via N_JOBS in Makefile and preset in .neuro/live.yamlparameters respectfully.
  • Run the master node with the cpu-small preset.
  • Auto-generate a NNI configuration file for the master node pointing at the workers.
  • Run the training process and automatically open the NNI web interface in your browser.
You can track experiment progress and intermediate results from this web UI. When the workers are done, you can get the final hyperparameter values and download the logs if needed.
NNI Hyperparameter Tuning GUI
Once you're done, you can shut down the workers and the master node by running
$ neuro-flow kill ALL