ERNI I Machine Learning

Defining the requirements for machine learning products

Complex software development projects traditionally incorporate requirements engineers who elicit and define the requirements of a piece of software by means of various tools. Although the agile process assigns requirements specification responsibilities to the product owner, who may delegate tasks to the developers, highly skilled requirements engineers are a much-solicited role (at ERNI). As many companies are deciding to launch machine learning projects, we want to discuss the requirements that must be considered and understood by all stakeholders.

Machine Learning Michael Schroeder

Michael Schroeder
ERNI Switzerland

 

Metrics matter

Developed and deployed machine learning (ML) approaches are evaluated by metrics that reflect the performance of the ML solution. Each requirement for a solution has a direct impact on the metric that should be used. When classifying, we must often gauge between classifying all wanted cases (high recall) or to be extremely sure when classifying a case (high precision). In other words, it is crucial to not mislabel; e.g., when labelling a customer as a loan defaulter, we should prioritise the precision metric over the recall metric as losing good customers may be more costly than not labelling a potential defaulter. While other and more complex metrics can be considered, it is important to assure that all stakeholders understand the impact of the metric in order to assert its interpretation.

 

Understanding the model

Furthermore, it is important to ask if we need to be able to explain what factors contributed to a certain decision or classification, e.g. in highly regulated markets or cases where human lives could be impacted by ML system outputs. Certain algorithms allow for good explainability, while others, e.g. neural networks, are referred to as black box procedures. In such cases, we may have to gauge explainability with more complex approaches, possibly with higher predictive power. Nevertheless, lot of research is underway on how we can shed some light into black box procedures.

 

Artificial intelligence is up for regulation

Related to the requirements on explainability, we need to take current and upcoming legal constraints into account. The European Union has set the tone with the introduction of the GDPR and the draft publication for the regulation of AI systems. Sector-specific requirements, such as the HIPAA for medical data, must be considered equally. As ML solutions are generally data hungry, the choice of use cases to pursue and the associated data strategy may require legal consulting as well as privacy and transparency mechanisms.

 

Data, data, data

More obvious requirements may fall under the category of data requirements. We need to make sure that the available and used data sample is unbiased and balanced, of good quality, complete and consistent. It may occur that certain data sources must be cleaned up in order to fulfil the requirements. Related to the data sourcing, requirements regarding how and from where data is sourced must be considered. Do we start to source from the live systems right away, creating the need to establish data pipelines? The opposing requirement may exist – that only deployable models shall be included in the live system, which implies an offline development and a data batching strategy. Influencing factors are, e.g. sensitivity of the data and the operational requirements for a live system.

 

 

ERNI I Machine Learning

 

 

Conclusion

In summary, when specifying the requirements of ML systems, there are a few particularities to keep an eye on. The exchange between data scientists, domain experts and legal entities is crucial to success, and the person taking the role of requirements engineer needs to be able to understand and keep in mind the technical aspects, the business goals and the interplay between these two. At ERNI we have an experienced team of business analysts, requirements engineers, data scientists and ML engineers, across multiple countries, with excellent communication skills which allow them to translate business requirements to technical requirements.

 

Important References:
Vogelsang and Borg [2019]
European Commission, April 21, 2021, IP/21/1682

News from ERNI

In our newsroom, you find all our articles, blogs and series entries in one place.

  • 27.09.2023.
    Newsroom

    Unveiling the power of data: Part III – Navigating challenges and harnessing insights in data-driven projects

    Transforming an idea into a successful machine learning (ML)-based product involves navigating various challenges. In this final part of our series, we delve into two crucial aspects: ensuring 24/7 operation of the product and prioritising user experience (UX).

  • 13.09.2023.
    Newsroom

    Exploring Language Models: An overview of LLMs and their practical implementation

    Generative AI models have recently amazed with unprecedented outputs, such as hyper-realistic images, diverse music, coherent texts, and synthetic videos, sparking excitement. Despite this progress, addressing ethical and societal concerns is crucial for responsible and beneficial utilization, guarding against issues like misinformation and manipulation in this AI-powered creative era.

  • 01.09.2023.
    Newsroom

    Peter Zuber becomes the new Managing Director of ERNI Switzerland

    ERNI is setting an agenda for growth and innovation with the appointment of Peter Zuber as Managing Director of the Swiss business unit. With his previous experience and expertise, he will further expand the positioning of ERNI Switzerland, as a leading consulting firm for software development and digital innovation.

  • data230.08.2023.
    Newsroom

    Unveiling the power of data: Part II – Navigating challenges and harnessing insights in data-driven projects

    The second article from the series on data-driven projects, explores common challenges that arise during their execution. To illustrate these concepts, we will focus on one of ERNI’s latest project called GeoML. This second article focuses on the second part of the GeoML project: Idea2Proof.

  • 16.08.2023.
    Newsroom

    Unveiling the power of data: Part I – Navigating challenges and harnessing insights in data-driven projects

    In this series of articles (three in total), we look at data-driven projects and explore seven common challenges that arise during their execution. To illustrate these concepts, we will focus on one of ERNI’s latest project – GeoML, dealing with the development of a machine learning algorithm capable of assessing road accident risks more accurately than an individual relying solely on their years of personal experience as a road user, despite limited resources and data availability.

     

  • 09.08.2023.
    Newsroom

    Collaborative robots revolutionising the future of work

    The future of work involves collaboration between robots and humans. After many years of integrating technology into work dynamics, the arrival of collaborative robots, or cobots, is a reality, boosting not only safety in the workplace but also productivity and efficiency in companies.

  • 19.07.2023.
    Newsroom

    When the lid doesn’t fit the container: User Experience Design as risk minimisation

    Struggling with a difficult software application is like forcing a lid onto a poorly fitting container. This article explores the significance of user experience (UX) in software development. Discover how prioritising UX improves efficiency and customer satisfaction and reduces risks and costs. Join us as we uncover the key to successful software applications through user-centric design.

  • 21.06.2023.
    Newsroom

    How does application security impact your business?

    With the rise of cyber threats and the growing dependence on technology, businesses must recognize the significance of application security as a fundamental pillar for protecting sensitive information and preserving operational resilience.