main.py, the path to the resulting serialized model is baked right into the code:
train.Dockerfilefor further use. The image will be based on a pre-built PyTorch image. The complete list of such images can be found on PyTorch DockerHub.
.in this case), use the build steps from
train.Dockerfile, and save the resulting image under
image:examples/mnist:trainin the Neu.ro registry. We can check the registry contents by using the following command:
neuro-extrasprovides a scaffolding for wrapping the model's code into a functional inference HTTP server that you can run in Neu.ro or any other container runtime.
seldon_model.pyis a Python module responsible for implementing the interface that Seldon Core expects. Typically, this module reads a serialized model from a mounted storage volume, deserializes this model, and uses it to predict on incoming data points.
seldon.Dockerfileis a predefined Dockerfile that assembles your code and the Seldon Core inference HTTP server for further use.
/storageto make it work. Another point to note is that we'll be running inference on CPU.
predictmethod. As we are dealing with image classification problem in this example, we would like our inference HTTP server to be able to recieve an image as binary data in
bytes, predict the classes, and return a JSON document with the resulting scores.
5000(the default port for the Seldon Core HTTP server) and mounts a storage volume with the serialized model to the path mentioned above.
https://example-mnist--user.jobs.neuro-ai-public.org.neu.ro/, we can now create a
curlcommand. Seldon Core HTTP server expects binary data sent as the
2has the highest score, as expected.
neuro-extrasallows creating the required resources easily:
curlcommand, but change the URL. It should lead to your K8S ingress gateway.
2and the setup is working properly.