kubectl
configured locally to be able to create all necessary K8S resources.main.py
, the path to the resulting serialized model is baked right into the code:train.Dockerfile
for further use. The image will be based on a pre-built PyTorch image. The complete list of such images can be found on PyTorch DockerHub..
in this case), use the build steps from train.Dockerfile
, and save the resulting image under image:examples/mnist:train
in the Neu.ro registry. We can check the registry contents by using the following command:neuro-extras
provides a scaffolding for wrapping the model's code into a functional inference HTTP server that you can run in Neu.ro or any other container runtime.seldon_model.py
is a Python module responsible for implementing the interface that Seldon Core expects. Typically, this module reads a serialized model from a mounted storage volume, deserializes this model, and uses it to predict on incoming data points.seldon.Dockerfile
is a predefined Dockerfile that assembles your code and the Seldon Core inference HTTP server for further use.seldon_model.py
:/storage
to make it work. Another point to note is that we'll be running inference on CPU.predict
method. As we are dealing with image classification problem in this example, we would like our inference HTTP server to be able to recieve an image as binary data in bytes
, predict the classes, and return a JSON document with the resulting scores.5000
(the default port for the Seldon Core HTTP server) and mounts a storage volume with the serialized model to the path mentioned above.curl
command later..jpg
image from the MNIST dataset: https://example-mnist--user.jobs.neuro-ai-public.org.neu.ro/
, we can now create a curl
command. Seldon Core HTTP server expects binary data sent as the binData
form field:2
has the highest score, as expected.neuro-extras
allows creating the required resources easily:curl
command, but change the URL. It should lead to your K8S ingress gateway.2
and the setup is working properly.