“An Algorithm Has No Sense of Tact”. With this title, a new book published a few days ago is causing quite a stir. It was written by Katharina Zweig, head of the Algorithm Accountability Lab and professor of informatics at the technical university (TU) in Kaiserslautern. She has been consulted by various German government ministries, is on the Artificial Intelligence committee of enquiry in the Bundestag and popular as a public speaker.

What is the scientist currently doing? “We are developing a software development process for algorithmic decision systems that integrates ethical aspects,” she explains. Algorithms and morality – how do they work together? Do data create something like truth? And if so, whose truth? That of a diverse society or that of old white men at the controls of power?

High error rates and programmed discrimination

In her book, the professor describes a frightening example: COMPAS is an AI software widely used in the USA that calculates how likely it is that a criminal will relapse. The exact working methods of its algorithms are not known, but it is a matter of classifying criminals according to certain characteristics. The learning software assigns these groups of people a value between 1 and 10, whereby a higher number in this score is accompanied by an increased probability of recidivism.

Tests – according to the author in her book – have shown that the error rate in predicting serious acts of violence is 75 percent. And the accuracy of the software’s predictions is also completely wrong in other respects in one-third of the cases. At the same time, the program predicts a high probability of recidivism for suspects who are conspicuously often dark-skinned. The program, which supposedly works objectively, has thus reproduced the racism of the judicial authorities towards black people, because it simply reproduces the existing discrimination.

Wherever software judges people or decides their participation in society, it gets complicated. Best examples are applications in the field of justice or HR. After all, how are algorithms supposed to map the human discretionary leeway available there? And can data sets be made more “moral” by collecting and analysing more and more data?

“Ethics by design” first asks for the meaning

Algorithm ethics tries to find socially acceptable answers to this controversial topic. Its goal: to consider ethical rules already during the development of intelligent software. Sarah Spiekermann, Professor of Business Ethics in Vienna, has created a catalogue of “value principles” together with around a thousand other experts from the global engineering association IEEE, and is focusing on “ethics by design”.

This approach describes a field of research in which the aim is to implement ethical processes in the construction, development and design of technologies and to consider their possible consequences right from the start. For this purpose, a so-called value-based system design must take place.

“This means that we ask the question at a very early stage in the development phases of technology: Why do we want this technology? What positive values do we set free for people and for society? Where do we have to be careful not to undermine existing values?” the scientist describes this approach. “If you do this very early in the development process, you can get a lot of good things off the ground.“

Software developers need to look beyond their own horizons

But also programmers with good intentions often miss the necessary care in software development according to the critical business information scientist. In agile improvement mode, they work towards a goal, sprint for sprint, but without asking why.

Maybe that’s why not better algorithms are in demand, but above all other developers who look beyond their own horizons. At the TU Kaiserslautern, Katharina Zweig is already trying during her training to build a bridge between computer engineering and society.

The “Socioinformatics” course, the first of its kind in Germany and established in 2013, teaches students programming as well as the basics of empirical social research and psychology, economics, law and also ethics.

News from ERNI

In our newsroom, you find all our articles, blogs and series entries in one place.

  • 22.11.2023.
    Newsroom

    Recognising trends: An insight into regression analysis

    Data plays a very important role in every area of a company. When it comes to data, a distinction is made primarily between operational data and dispositive data. Operational data play an important role, especially in day-to-day business. However, they are not nearly as relevant as dispositive data. This is because these data are collected over a longer period of time and provide an initial insight into the history or the past.

  • 08.11.2023.
    Newsroom

    Why do we need digital transformation for medical devices?

    For hospitals, it is not up for discussion as to whether they want to digitalise. The increasing age of the population in western countries and the progressive shortage of medical professionals mean that without digitalisation, the healthcare system will not be able to provide the quality that patients want in the future.

  • 25.10.2023.
    Newsroom

    Mastering the challenges of mobile app testing: Strategies for efficient quality assurance

    Discover the unique challenges faced in testing mobile applications and learn how to overcome them effectively. From selecting suitable devices and operating systems to leveraging cloud-based test platforms, test automation and emulators, this article provides seven essential strategies for optimising your mobile app testing process.

  • 11.10.2023.
    Newsroom

    Incorporating classical requirements engineering methods in agile software development for a laboratory automation system

    Traditional agile methodologies can sometimes struggle to accommodate the complexity and regulatory requirements of laboratory automation systems, leading to misalignment with stakeholder needs, scope creep, and potential delays. The lack of comprehensive requirements documentation can result in ambiguous expectations and hinder effective communication among cross-functional teams.

  • 27.09.2023.
    Newsroom

    Unveiling the power of data: Part III – Navigating challenges and harnessing insights in data-driven projects

    Transforming an idea into a successful machine learning (ML)-based product involves navigating various challenges. In this final part of our series, we delve into two crucial aspects: ensuring 24/7 operation of the product and prioritising user experience (UX).

  • 13.09.2023.
    Newsroom

    Exploring Language Models: An overview of LLMs and their practical implementation

    Generative AI models have recently amazed with unprecedented outputs, such as hyper-realistic images, diverse music, coherent texts, and synthetic videos, sparking excitement. Despite this progress, addressing ethical and societal concerns is crucial for responsible and beneficial utilization, guarding against issues like misinformation and manipulation in this AI-powered creative era.

  • 01.09.2023.
    Newsroom

    Peter Zuber becomes the new Managing Director of ERNI Switzerland

    ERNI is setting an agenda for growth and innovation with the appointment of Peter Zuber as Managing Director of the Swiss business unit. With his previous experience and expertise, he will further expand the positioning of ERNI Switzerland, as a leading consulting firm for software development and digital innovation.

  • data230.08.2023.
    Newsroom

    Unveiling the power of data: Part II – Navigating challenges and harnessing insights in data-driven projects

    The second article from the series on data-driven projects, explores common challenges that arise during their execution. To illustrate these concepts, we will focus on one of ERNI’s latest project called GeoML. This second article focuses on the second part of the GeoML project: Idea2Proof.