Deploying to production
The Azure Model Management can turn our pre-made Docker image into a scalable deployment in a few clicks. But the cluster environment needs to be set up first:
az ml env setup --cluster -l westeurope -n amlwclusterenv -g AMLW --agent-vm-size Standard_A4_v2
This will provision a new set of environment resources in the cloud. This operation can take up to 20 minutes. Note the –agent-agent-size parameter – it controls what kind of VMs will be used to execute the model. Depending on the demands of your model, you might want to specify other types of machines.
The list of valid size choices also depends on the pricing tier of your Model Management account – if you chose to use the DevTest tier, you are limited to only a few choices. The Standard_A4_v2 should work, however.
Once the environment is ready, navigate into the Model Management section in the Azure portal. If you followed the previous steps in this tutorial, you should have a deployable image already created:
Click on the image you want to deploy and hit the “Create Service” button. You will be brought to a settings screen for the service:
Pick a subscription (if you have more than one) and select the appropriate environment. Its Cluster Type should be ACS:
You can fill out additional settings below, including under the Advanced Settings – stuff such as minimum/maximum number of running replicas and scaling rules. When you are satisfied, hit Create.
The operation can take some time – the image (remember, it can be several GB) is being downloaded on all the agent machines and everything is being registered with Kubernetes. After everything succeeds, you can display the service’s details:
This overview tells you the address where the service lives, including the authorization keys you need to successfully invoke the it.
You can use Postman to test the live deployment – specify the URL and payload and add Bearer token authorization. Use one of the displayed keys as the token.
And that’s all, folks! This marks the end of the journey – from a locally-trained ML model all the way to scalable cloud deployment in just a few lines of code and command line.
Conclusion
And that’s all, folks. We have arrived at our destination – we started with a machine learning model and now we have a way to host, manage, and consume it from our applications. The road was not as straightforward as it could have been, but the Azure ML Workbench is still in preview, so maybe cut it some slack.
I hope this series of articles was as interesting to read as it was interesting to write (and research). Thanks for sticking with me throughout the series – happy learning!
Resources
Installing the Azure ML Workbench: https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-installation
Configuring the local environment for deployment: https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/deployment-setup-configuration
Deploying a model: https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/model-management-service-deploy
Read previous articles:
