Michael Schroeder
ERNI Switzerland
Metrics matter
Developed and deployed machine learning (ML) approaches are evaluated by metrics that reflect the performance of the ML solution. Each requirement for a solution has a direct impact on the metric that should be used. When classifying, we must often gauge between classifying all wanted cases (high recall) or to be extremely sure when classifying a case (high precision). In other words, it is crucial to not mislabel; e.g., when labelling a customer as a loan defaulter, we should prioritise the precision metric over the recall metric as losing good customers may be more costly than not labelling a potential defaulter. While other and more complex metrics can be considered, it is important to assure that all stakeholders understand the impact of the metric in order to assert its interpretation.
Understanding the model
Furthermore, it is important to ask if we need to be able to explain what factors contributed to a certain decision or classification, e.g. in highly regulated markets or cases where human lives could be impacted by ML system outputs. Certain algorithms allow for good explainability, while others, e.g. neural networks, are referred to as black box procedures. In such cases, we may have to gauge explainability with more complex approaches, possibly with higher predictive power. Nevertheless, lot of research is underway on how we can shed some light into black box procedures.
Artificial intelligence is up for regulation
Related to the requirements on explainability, we need to take current and upcoming legal constraints into account. The European Union has set the tone with the introduction of the GDPR and the draft publication for the regulation of AI systems. Sector-specific requirements, such as the HIPAA for medical data, must be considered equally. As ML solutions are generally data hungry, the choice of use cases to pursue and the associated data strategy may require legal consulting as well as privacy and transparency mechanisms.
Data, data, data
More obvious requirements may fall under the category of data requirements. We need to make sure that the available and used data sample is unbiased and balanced, of good quality, complete and consistent. It may occur that certain data sources must be cleaned up in order to fulfil the requirements. Related to the data sourcing, requirements regarding how and from where data is sourced must be considered. Do we start to source from the live systems right away, creating the need to establish data pipelines? The opposing requirement may exist – that only deployable models shall be included in the live system, which implies an offline development and a data batching strategy. Influencing factors are, e.g. sensitivity of the data and the operational requirements for a live system.
Conclusion
In summary, when specifying the requirements of ML systems, there are a few particularities to keep an eye on. The exchange between data scientists, domain experts and legal entities is crucial to success, and the person taking the role of requirements engineer needs to be able to understand and keep in mind the technical aspects, the business goals and the interplay between these two. At ERNI we have an experienced team of business analysts, requirements engineers, data scientists and ML engineers, across multiple countries, with excellent communication skills which allow them to translate business requirements to technical requirements.
Important References:
Vogelsang and Borg [2019]
European Commission, April 21, 2021, IP/21/1682
