English
Neu.ro Core
Neu.ro Toolbox
Creating a Cluster
Introduction
Neu.ro lets you create a cluster in any of the three major cloud providers - Amazon Web Services, Google Cloud Platform, and Azure. After you sign up with a cloud provider, you have to share your service account information and the configuration you need with our team. We will then set up a cluster on your behalf, and install Neu.ro on your cloud for you.
To set up a Neu.ro cluster, you have to do the following:
- 1.Set up a cloud environment:
- 2.Prepare a cluster configuration YAML file.
- 3.Share the configuration YAML file with us. Once we receive the YAML file, we will set up and run the cluster (usually completed within one business day). You can start adding your users to the cluster while it is being set up.
Apart from the process mentioned above, there are other methods of setup:
- Set up Neu.ro on an existing cluster provided by AWS, GCP, or Azure. This requires the configuration of the node pool.
- Set up Neu.ro on an existing cluster provided by other cloud service providers.
- Set up Neu.ro on-premise (or “bare metal”).
Cluster configuration YAML
You must create a project/VPC/resource group and a service account with all required permissions before you can start preparing a cluster configuration YAML file. The YAML file is used by the Neu.ro team to set up and run the cluster. You should use the
neuro admin generate-cluster-config command to generate the YAML file. It is an interactive tool that generates a valid configuration file with the default node configuration based on your responses.The command prompts for the following information in order to generate the configuration file:
Prompt
Description
Cluster Type
Enter cloud service provider - AWS, GCP, Azure
GCP Project Name or AWS VPC ID or Azure subscription ID
- For GCP, enter the name of the project.
- For AWS, enter the VPC ID you created in the previous step.
- For Azure, enter the Azure subscription ID you created in the previous step.
Service Account Key File (.json)
Or
AWS Profile Name
Or Azure information
- For GCP, enter the path to the service account key file. For information on service accounts, see Understanding service accounts.
- For Azure, enter the following access information:
- Azure Files storage size (GB)
Below is a sample command for GCP:
1
> neuro admin generate-cluster-config
2
Select cluster type (aws, gcp, azure): gcp
3
GCP project name: My Project
4
Service Account Key File (.json): GCP_User.json
5
Cluster config cluster.yml is generated.
Copied!
The command creates the
cluster.yml file that includes the information required by the Neu.ro team to set up your cluster. Here is a sample cluster.yml generated for a new GCP cluster:1
type: gcp
2
location_type: multi_zonal
3
region: us-central1
4
zones:
5
- us-central1-a
6
- us-central1-c
7
project: My Project
8
credentials:
9
auth_provider_x509_cert_url: https://www.googleapis.com/oauth2/v1/certs
10
auth_uri: https://accounts.google.com/o/oauth2/auth
11
client_email: [email protected]
12
client_id: '105087309181394151560'
13
client_x509_cert_url: https://www.googleapis.com/robot/v1/metadata/x509/johndoe%40intricate-web-236410.iam.gserviceaccount.com
14
private_key: ...
15
private_key_id: ...
16
project_id: intricate-web-236410
17
token_uri: https://oauth2.googleapis.com/token
18
type: service_account
19
node_pools:
20
- id: n1_highmem_8
21
min_size: 1
22
max_size: 4
23
- id: n1_highmem_8_1x_nvidia_tesla_k80
24
min_size: 1
25
max_size: 4
26
- id: n1_highmem_8_1x_nvidia_tesla_v100
27
min_size: 0
28
max_size: 1
29
storage:
30
id: standard
31
capacity_tb: 4
Copied!
The file contains a default nodes pools configuration that is used as a starting point:
- 1 non-preemptive node with K80
- 1 non-preemptive node with V100
- 4 non-preemptive non-GPU nodes
- and 4 Tb of standard storage
The file will create a cluster with the following presets:
- cpu-small,
- cpu-large,
- gpu-small (a node with K80), and
- gpu-large (a node with V100).
To get information about available options for each of the cloud providers, run:
neuro admin show-cluster-options .You can further update the
cluster.yml file as per your requirements before you send it to us. If you have any issues updating the file, then contact us. Once you are done updating the configuration file, you should send the cluster.yml file to the Neu.ro team for the cluster setup using the command:neuro admin add-cluster <path/to/config>Once we receive the YAML file, we will set up and run the cluster (usually completed within one business day). After the command is run, you become the admin of the cluster. You can start adding users to the cluster as soon as you run the command above.
Last modified 1yr ago
Copy link