Marek Linka, Senior Software Developer

Marek Linka

Senior Software Developer

Operationalizing machine learning models 3/3 – Production-grade, scalable deployment

The previous two articles in this series introduced the Azure Machine Learning Workbench and walked us through the process of wrapping our ML model in a self-contained Docker image. Now is the time to take this image and deploy it into a production-grade environment.

Deploying to production

The Azure Model Management can turn our pre-made Docker image into a scalable deployment in a few clicks. But the cluster environment needs to be set up first:

az ml env setup --cluster -l westeurope -n amlwclusterenv -g AMLW --agent-vm-size Standard_A4_v2

This will provision a new set of environment resources in the cloud. This operation can take up to 20 minutes. Note the –agent-agent-size parameter – it controls what kind of VMs will be used to execute the model. Depending on the demands of your model, you might want to specify other types of machines.

The list of valid size choices also depends on the pricing tier of your Model Management account – if you chose to use the DevTest tier, you are limited to only a few choices. The Standard_A4_v2 should work, however.

Once the environment is ready, navigate into the Model Management section in the Azure portal. If you followed the previous steps in this tutorial, you should have a deployable image already created:

Click on the image you want to deploy and hit the “Create Service” button. You will be brought to a settings screen for the service:

Pick a subscription (if you have more than one) and select the appropriate environment. Its Cluster Type should be ACS:

You can fill out additional settings below, including under the Advanced Settings – stuff such as minimum/maximum number of running replicas and scaling rules. When you are satisfied, hit Create.

The operation can take some time – the image (remember, it can be several GB) is being downloaded on all the agent machines and everything is being registered with Kubernetes. After everything succeeds, you can display the service’s details:

This overview tells you the address where the service lives, including the authorization keys you need to successfully invoke the it.

You can use Postman to test the live deployment – specify the URL and payload and add Bearer token authorization. Use one of the displayed keys as the token.

And that’s all, folks! This marks the end of the journey – from a locally-trained ML model all the way to scalable cloud deployment in just a few lines of code and command line.

Conclusion

And that’s all, folks. We have arrived at our destination – we started with a machine learning model and now we have a way to host, manage, and consume it from our applications. The road was not as straightforward as it could have been, but the Azure ML Workbench is still in preview, so maybe cut it some slack.

I hope this series of articles was as interesting to read as it was interesting to write (and research). Thanks for sticking with me throughout the series – happy learning!

Resources

Installing the Azure ML Workbench: https://docs.microsoft.com/en-us/azure/machine-learning/service/quickstart-installation

Configuring the local environment for deployment: https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/deployment-setup-configuration

Deploying a model: https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/model-management-service-deploy

Read previous articles:

News from ERNI

In our newsroom, you find all our articles, blogs and series entries in one place.

> Load more

ERNI Schweiz

Casinoplatz 2

3011 Bern

Phone: +41 58 268 12 00

Email: [email protected]

ERNI Suisse

Bâtiment L

Route des Acacias 43

1227 Geneva

Phone: +41 58 268 11 03

Email: [email protected]

ERNI Suisse

Voie du Chariot 3

1003 Lausanne

Phone: +41 58 268 11 03

Email: [email protected]

ERNI Schweiz

Brünigstrasse 18

6005 Lucerne

Phone: +41 58 268 11 03

Email: [email protected]

ERNI Schweiz

Geschäftshaus Airgate

Thurgauerstrasse 40

8050 Zürich

Phone: +41 58 268 12 00

Email: [email protected]

ERNI Deutschland

Trakehner Str. 7-9

60487 Frankfurt am Main

Phone: +49 69 79 53 28 01

Email: [email protected]

ERNI Deutschland

Design Offices München – Arnulfpark

Luise-Ullrich-Str. 20

80636 München

Phone: +49 89 55 06 28 208

Email: [email protected]

ERNI Slovakia

Ševčenkova 34

851 01 Bratislava

Phone: +421 2 32 55 37 37

Email: [email protected]

ERNI España

Edificio El Triangle

Plaça Catalunya 1-4, 3º planta, Módulo A y B

08002 Barcelona

Phone: +34 93 667 77 76

Email: [email protected]

ERNI España

Carrer Pallars, 208, Bajos

08005 Barcelona

Phone: +34 93 667 77 76

Email: [email protected]

ERNI España

Calle de Alfonso XII 62

Oficina 3101

28014 Madrid

Phone: +34 901 848 787

Email: [email protected]

ERNI España

Sant Cugat ERNI Office

Plaça Xavier Cugat, 2 EDIF B Planta Baja

08174 Sant Cugat del Vallès

Phone: +34 93 667 77 76

Email: [email protected]

ERNI Romania

Calea Dorobantilor no. 98-100

3rd floor, in Olimpia Business Center

400609 Cluj-Napoca

Phone: +40 744 319 228

Email: [email protected]

ERNI Singapore

7 Straits View

Marina One East Tower #05-01

Singapore 018936

Phone: +65 9161 9863

Email: [email protected]

ERNI Philippines

9th Floor, 500 Shaw Zentrum Building

500 Shaw Boulevard

Mandaluyong City, Philippines 1555

Phone: +63 2 531 59 82

Email: [email protected]