Deploying to production

The Azure Model Management can turn our pre-made Docker image into a scalable deployment in a few clicks. But the cluster environment needs to be set up first:

az ml env setup --cluster -l westeurope -n amlwclusterenv -g AMLW --agent-vm-size Standard_A4_v2

This will provision a new set of environment resources in the cloud. This operation can take up to 20 minutes. Note the –agent-agent-size parameter – it controls what kind of VMs will be used to execute the model. Depending on the demands of your model, you might want to specify other types of machines.

The list of valid size choices also depends on the pricing tier of your Model Management account – if you chose to use the DevTest tier, you are limited to only a few choices. The Standard_A4_v2 should work, however.

Once the environment is ready, navigate into the Model Management section in the Azure portal. If you followed the previous steps in this tutorial, you should have a deployable image already created:

Click on the image you want to deploy and hit the “Create Service” button. You will be brought to a settings screen for the service:

Pick a subscription (if you have more than one) and select the appropriate environment. Its Cluster Type should be ACS:

You can fill out additional settings below, including under the Advanced Settings – stuff such as minimum/maximum number of running replicas and scaling rules. When you are satisfied, hit Create.

The operation can take some time – the image (remember, it can be several GB) is being downloaded on all the agent machines and everything is being registered with Kubernetes. After everything succeeds, you can display the service’s details:

This overview tells you the address where the service lives, including the authorization keys you need to successfully invoke the it.

You can use Postman to test the live deployment – specify the URL and payload and add Bearer token authorization. Use one of the displayed keys as the token.

And that’s all, folks! This marks the end of the journey – from a locally-trained ML model all the way to scalable cloud deployment in just a few lines of code and command line.


And that’s all, folks. We have arrived at our destination – we started with a machine learning model and now we have a way to host, manage, and consume it from our applications. The road was not as straightforward as it could have been, but the Azure ML Workbench is still in preview, so maybe cut it some slack.

I hope this series of articles was as interesting to read as it was interesting to write (and research). Thanks for sticking with me throughout the series – happy learning!


Installing the Azure ML Workbench:

Configuring the local environment for deployment:

Deploying a model:

Read previous articles:

News from ERNI

In our newsroom, you find all our articles, blogs and series entries in one place.

  • 22.11.2023.

    Recognising trends: An insight into regression analysis

    Data plays a very important role in every area of a company. When it comes to data, a distinction is made primarily between operational data and dispositive data. Operational data play an important role, especially in day-to-day business. However, they are not nearly as relevant as dispositive data. This is because these data are collected over a longer period of time and provide an initial insight into the history or the past.

  • 08.11.2023.

    Why do we need digital transformation for medical devices?

    For hospitals, it is not up for discussion as to whether they want to digitalise. The increasing age of the population in western countries and the progressive shortage of medical professionals mean that without digitalisation, the healthcare system will not be able to provide the quality that patients want in the future.

  • 25.10.2023.

    Mastering the challenges of mobile app testing: Strategies for efficient quality assurance

    Discover the unique challenges faced in testing mobile applications and learn how to overcome them effectively. From selecting suitable devices and operating systems to leveraging cloud-based test platforms, test automation and emulators, this article provides seven essential strategies for optimising your mobile app testing process.

  • 11.10.2023.

    Incorporating classical requirements engineering methods in agile software development for a laboratory automation system

    Traditional agile methodologies can sometimes struggle to accommodate the complexity and regulatory requirements of laboratory automation systems, leading to misalignment with stakeholder needs, scope creep, and potential delays. The lack of comprehensive requirements documentation can result in ambiguous expectations and hinder effective communication among cross-functional teams.

  • 27.09.2023.

    Unveiling the power of data: Part III – Navigating challenges and harnessing insights in data-driven projects

    Transforming an idea into a successful machine learning (ML)-based product involves navigating various challenges. In this final part of our series, we delve into two crucial aspects: ensuring 24/7 operation of the product and prioritising user experience (UX).

  • 13.09.2023.

    Exploring Language Models: An overview of LLMs and their practical implementation

    Generative AI models have recently amazed with unprecedented outputs, such as hyper-realistic images, diverse music, coherent texts, and synthetic videos, sparking excitement. Despite this progress, addressing ethical and societal concerns is crucial for responsible and beneficial utilization, guarding against issues like misinformation and manipulation in this AI-powered creative era.

  • 01.09.2023.

    Peter Zuber becomes the new Managing Director of ERNI Switzerland

    ERNI is setting an agenda for growth and innovation with the appointment of Peter Zuber as Managing Director of the Swiss business unit. With his previous experience and expertise, he will further expand the positioning of ERNI Switzerland, as a leading consulting firm for software development and digital innovation.

  • data230.08.2023.

    Unveiling the power of data: Part II – Navigating challenges and harnessing insights in data-driven projects

    The second article from the series on data-driven projects, explores common challenges that arise during their execution. To illustrate these concepts, we will focus on one of ERNI’s latest project called GeoML. This second article focuses on the second part of the GeoML project: Idea2Proof.