Gábor Patassy, Senior Software Developer

Techletter: How to speed up Data Science tasks in Python using Dask

Nowadays there is a great deal of hype around Data science. But what exactly is it? Let me take a stab at a basic definition. And also explain why Python is the programming language to go for in this field and why Dask comes into play.

“Data science is a multi-disciplinary field that uses scientific methods, processes, algorithms and systems to extract knowledge and insights from data in various forms.”

 

Source: OECD Governance

 

There are some requirements that need to be met for those wanting to do Data science projects. Data science requires some software development skills, mathematics and of course domain knowledge. What tasks are solved by Data science? What it covers is mainly machine learning, data analysis, data cleaning and financial analysis.

Why Python?

In recent years Python has gained a lot of momentum. Some people call it “The Swiss Army Knife of Coding”. Python is a general-purpose language with a fast learning curve and it has been designed with readability as one of the most important aspects in mind. In my view, the two main reasons for Python’s popularity are:

  • Simplicity and time to market
  • Large open source community

Python has a great number of scientific computing packages. The most important ones can be found in the following picture:

 

Source: Quansight

 

Today we will mainly focus on Pandas and NumPy (and later their extended versions in Dask).

Pandas is a Python package providing fast, flexible and expressive data structures designed to make working with “relational” or “labelled” data both easy and intuitive. Pandas is mainly used for handling tabular data.

  • NumPy – Contains a powerful N-dimensional array object, tools for integrating C/C++ and Fortran code.

Usually people think that Python is slow. This is true for standard Python objects, however NumPy and Pandas libraries are mostly written in C/Fortran, which makes them significantly faster.

The next graph shows some statistics about the most visited Python-related questions on Stack Overflow:

 

Stack Overflow Traffic ERNI
Source: Stack Overflow Blog

 

You can see that Pandas and NumPy are increasing in popularity.

If they are so popular and fast, what’s the problem with them?

Both packages are designed for:

  • in-memory data – i.e. you can’t work with datasets that are larger than your RAM
  • single core execution by default.

*Note: I am lying a little bit here; some methods in NumPy are multithreaded by default.

Sometimes when trying to work with large datasets in Pandas, your machine just freezes, and you’re stuck waiting for minutes just to calculate a simple average over a column in your data frame.

This gets really annoying when you have a working solution in Python, but you are not able to scale it up. Of course you can always buy more RAM, but for the speedup, you need to reach out for other solutions. Sometimes those solutions would require rewriting the whole application from scratch. That is definitely not something your stakeholders want to hear.

This is where Dask comes into play

Dask is an open source project which gives you the possibility to run operations in parallel over NumPy arrays or Pandas DataFrames, and chunk your datasets into smaller pieces which can be stored on multiple worker nodes.

Dask provides high-level Array, Bag and DataFrame collections that mimic NumPy, lists and Pandas, but can operate in parallel on datasets that don’t fit into main memory. Dask’s high-level collections are alternatives to NumPy and Pandas for large datasets.

Don’t worry, Dask is not another open source project maintained by 2-3 people. Anaconda, Inc. stands behind it – the maintainer of the most popular Python distribution for scientific computing.

Advantages of Dask:

  • Manipulating large datasets, even when those datasets don’t fit in memory
  • Accelerating long computations by using many cores
  • API is almost the same as for NumPy and Pandas
  • With minimal changes in code, you can run your existing code
  • Dask can be scaled up very easily to cluster
  • On a local machine, you can parallelize using Multithreading or Multiprocessing

Dask Collections

  • Dask array
    o Internally coordinates many NumPy arrays
    o Co-evolves with NumPy
    o Supports reductions, blockwise, overlapping, linear algebra
    o Dask arrays coordinate many NumPy arrays arranged into a grid. These NumPy arrays may live on disk or on other machines.
Source: towardsdatascience.com

 

  • Dask DataFrames – scalable implementation of Pandas
    o Internally coordinates many Pandas DataFrames
    o Co-evolves with Pandas
    o Dask DataFrames coordinate many Pandas DataFrames arranged along the index

 

Source: towardsdatascience.com

 

  • Dask Bags
    o A Dask Bag is able to store and process collections of Pythonic objects that are unable to fit into memory. Dask Bags are great for processing logs and collections of json documents.
  • Dask delayed
    o Dask is not limited to NumPy and pandas. You can parallelize custom Python code.
    o With Dask Delayed each function call is queued, added to an execution graph and scheduled.
  • Dask ML
    o Scalable version of Python’s Scikit-Learn library for machine learning.

Syntax difference between NumPy and Dask Array

Numpy array:

 

import numpy as np
x
= np.random.random(10000, 10000)

 

Dask array:

import dask.array as da
x
= da.random.random((10000, 10000), chunks=(1000, 1000))

 

For the Dask array, you need to provide the chunk size. As a rule of thumb, try to set the chunk size so that the number of chunks will be N times the number of cores.

So how does Dask work?

Source: dask.pydata.org

 

The API of the selected Collection produces a Task graph. As all the operations are lazy loaded, nothing is executed at that moment. Once you run the computation, the scheduler executes the operations from the task graph.

The scheduler can reside on your local machine and use a thread or a process pool, or it can reside on a distributed cluster.

If we choose a distributed scheduler, our network system will look like the following picture (actually, the local system is very similar, but there the workers represent the cores of our CPU).

 

Source: blog.dask.org

There are 3 main components:

  • Scheduler – coordinates the whole workflow
  • Workers – distributed work and storage – computation, holds data in distributed memory
  • Clients – interfaces with the users through the API

How do these graphs work?

To understand the concept, let me explain Dask delayed first. The delayed method decorates your functions so that they operate lazily. Rather than executing your function immediately, it will defer execution, placing the function and its arguments into a task graph. Dask array, DataFrame uses delayed under the hood. In our example, we will use Dask delayed on some simple Python operations.

This example takes number 1 increments it, takes number 2 increments it, and then at the end it adds the two numbers together (i.e. the result should be 5).

Here is the Python code:

import dask
x = dask.delayed(inc)(1)
y = dask.delayed(inc)(2)
z = dask.delayed(add)(x, y)
z.vizualize()

TL 2 2019 Info8
Source: http://docs.dask.org/en/latest

The picture shows the generated task graph. As you can see, Dask was so smart that it identified that the two increment operations could be run in parallel.

If we want to calculate the result, we need to call the  .compute() method over the Dask object.

Of course in real life we get much more complicated task graphs:

TL 2 2019 Info9
Source: https://matthewrocklin.com/blog/

 

The task graph from above is still a simple one; normally you can get graphs with millions of operations.

Before executing the .compute() method, you might consider visualizing the task graph. By looking at it, you can identify potential bottlenecks, check if you are indeed parallelizing, or find places in the code where the tasks depend on each other (which can lead to a great deal of communication). Don’t try to use delayed on every function, and always check the task graph, otherwise you can get even worse results than the original solution.

How to parallelize with Dask

One important thing to note before you start parallelizing:

The key to accelerating your code … is understanding why it’s slow in the first place

As I said earlier for local Dask, you can switch between multithreading and multiprocessing. Multithreading in Python works differently than in other languages. Without going into too many details, the Global Interpreter Lock (GIL) stops multiple Python threads from operating in parallel, i.e. only one thread can be in a state of execution at any point in time.

Fortunately, most of the code in NumPy and Pandas is written in C and bypasses the GIL. If you use multiprocessing, the code can be run in parallel on the different cores of the CPU – however, creating processes involves serialization/deserialization of the sent into them, so there is an additional overhead. For I/O bound tasks, you should use multithreading; for CPU bound tasks (e.g. numeric computations), use multiprocessing. Of course you can always switch between the two in Dask, to see which one is better for your application.

And then there are operations which are very hard/impossible to parallelize. For example, operations that rearrange the full dataset are hard to do in parallel.

Which operations rearrange the datasets?

  • DataFrames: set index, groupby-apply operations, table joins
  • Arrays: rechunking, FFT analysis

Let’s imagine the following workflow:

  1. Reading csv files from a disk into a DataFrame
  2. Setting the index to different column
  3. Resampling the dataset
  4. Plotting the result

Even if we try to parallelize this workflow with Dask, we won’t get too much improvement. Why?

  • Reading from disk is still the same operation
  • Setting the index rearranges the dataset – it involves a large piece of interaction between the workers
  • Parsing the csv and resampling the dataset are the two places where we can gain some speed

Of course if you set up a large enough cluster, you can get some serious improvement, but the cost of the infrastructure will be much higher.

This example shows us that in certain cases, Dask is not ideal for speeding up the process (of course it still can be used for handling larger-than-memory datasets). But before you start with parallelization, you have to be sure that this is exactly what your application needs.

If you decide to do parallelization, chunking strategy is the next thing to consider. You should be very careful about how you chunk your dataset:

  • Too large chunks – It’s possible that you wouldn’t use all your workers, and the involved workers wouldn’t be able to handle the load because the individual chunks are too large.
  • Too small chunks – Each chunk processing involves some additional tasks like communication overhead, serializing or deserializing. If you have too many chunks, these additional tasks could slow down the processing.

With more complex algorithms, the optimization becomes much harder. If there are tasks that depend on each other, or if the graph size is too large, it is hard to find an optimal solution.

Fortunately, Dask has its own diagnostics tool, where you can see what the individual workers are doing in real time, how they are communicating with each other, and spot some places with blocking code.

The web interface of the diagnostic tool is launched on the machine where the scheduler is installed.

It contains the following plots:

  • Progress – The interface shows the progress of the various computations as well as the exact number completed. Each bar is assigned a colour according to the function being run. On the left, the lighter shade is the number of tasks that have both been completed and released from memory.
Source: http://distributed.dask.org/en/latest/web.html
  • Memory – The interface shows the relative memory use of each function with a horizontal bar sorted by function name.
  • Task stream – The task stream plot shows when tasks complete on which workers. Worker cores are on the y-axis and time is on the x-axis. As a worker completes a task, its start and end times are recorded and a rectangle is added to this plot accordingly.

 

Source: http://distributed.dask.org/en/latest/web.html
  • Resources – The resources plot shows the average CPU and memory use over time as well as average network traffic.
Source: http://distributed.dask.org/en/latest/web.html

Dask vs. Spark

  • Spark is a big data ecosystem that has been used for many years. It is much larger than Dask; however, there are cases where Dask beats Spark.
  • Spark DataFrames will be much better when you have large SQL-style queries, where their query optimizer can kick in.
  • Dask DataFrames will be much better when queries go beyond typical database queries. This happens most often in time series, random access, and other complex computations.
  • Spark will integrate better with JVM and data engineering technology. Spark is its own ecosystem.
  • Dask will integrate better with Python code. Dask is designed to integrate with other libraries and pre-existing systems.
  • Spark does not include support for multi-dimensional arrays natively.
  • Dask fully supports the NumPy model for scalable multi-dimensional arrays. If your project highly relies on NumPy, you should consider using Dask.

Disadvantages of Dask

  • Dask is not SQL – it doesn’t optimize complex queries.
  • It is not as fast as MPI – communication overhead is worse.
  • Dask targets Python and associated languages.
  • Sometimes you don’t even need parallelism.
  • The API is similar to Pandas and NumPy. However, some of the functions are missing, and to bypass the functionality relevant part of the code that needs to be modified.

Enough theory; let’s take a look at a real-life example

For my research, I decided to implement a Monte Carlo simulation for finance applications, as it is highly parallelizable, and used very frequently in the world of finance.
What is a Monte Carlo simulation?

The Monte Carlo model allows researchers to run multiple trials and define all potential outcomes of an event or investment. Together, they create a probability distribution or risk assessment for a given investment or event.

A Monte Carlo analysis is a multivariate modelling technique. All multivariate models can be thought of as complex “what if?” scenarios. Research analysts use them to forecast investment outcomes, to understand the possibilities surrounding their investment exposures, and to better mitigate risks.

So, I started with a solution in NumPy. It works, but I got into troubles very quickly. Some products are complex (containing many sub-products) and the forecasting interval is more than 10 years. The resulting dataset can grow to more than 20 GB, and the computation is slow.

  • First I tried a Local Dask implementation using all 8 cores of my machine. As the computation is more CPU bound, I achieved the best results with multiprocessing. I had to run multiple experiments to find the optimal number of chunks.
  • Then I went to set up a cluster. Using Kubernetes and Google Cloud, I was able to set it up in less than 10 minutes.

The following graph shows the results of parallelization using Dask.

As you can see, Local Dask with multiprocessing gives good enough results, but it has to be noted that the clustered version used only 3 nodes with a standard computer, which was enabled in my subscription program. If I had set up a larger cluster with more performant machines, I would have achieved even better results.

Conclusion

Dask is a very promising technology backed by a large company. It simplifies the parallelization of existing Python-based Data science workflows. The developer team behind Dask is currently working on supporting GPU-based computation and parallelization, and the integration of just-in-time compilation library Numba.

News from ERNI

In our newsroom, you find all our articles, blogs and series entries in one place.

  • 17.04.2019.
    Newsroom

    Joseph Cadag: One day at ERNI Philippines

    Have you ever wondered what a day at ERNI looks like? We are starting a new video series about our colleagues and their working day.
  • unit economics in software development17.04.2019.
    Newsroom

    Unit Economics and Features

    Unit economics, a term from business meaning the fundamental minimum piece of your business that you can measure to understand where your revenue comes from, helps companies stay in control of their return of investments (ROIs).
  • 17.04.2019.
    Newsroom

    Azure Pipelines Agent Deployment

    This article can help you set up fully automated deployment of Azure DevOps build agents. It explains step-by-step instructions on how to set up automated deployment of self-hosted agents using Azure DevOps Pipelines.
  • 16.04.2019.
    Newsroom

    Social Project: ERNI Scholars Site Visit

    Helping someone to see them become better is not an easy task. You need to exert all your efforts, attention and time to achieve it.
  • 15.04.2019.
    Newsroom

    ERNI Hero Teams protect the nature

    The winners of the internal ERNI Hero Team contest decided to use their reward for a good purpose – to visit the Haribon Foundation at Lumban for seedlings preparation.  
  • ERNI Sarah Javier coffeee painting workshop12.04.2019.
    Newsroom

    ERNI Community: Coffee Painting workshop at ERNI Philippines

    Who says coffee is only good for drinking? It’s now good for painting too! Anyone can enjoy the art of coffee painting. This has encouraged me to start some Coffee Painting Workshops.
  • predictive maintenance solution by ERNI10.04.2019.
    Newsroom

    Step-by-step

    What does a Predictive Maintenance solution look like?
  • 03.04.2019.
    Newsroom

    Positive vibes at the Polymesse

    On 2 April 2019, we had the pleasure to attend the ETH Polymesse job fair in Zurich.
  • 01.04.2019.
    Newsroom

    Performance Days 2019: There is no better time to take a deep breath and slowly exhale

    As a habit already around mid of March, also this year the ERNI Performance Days (EPD) took place near the Bratislava site – this time in a recreational area of the city of Senec. Between 14 – 16 March the crew spent two days fully packed with workshop sessions developing technical as well as soft skills, networking and some sports and leisure activities.
  • xamarin ERNI01.04.2019.
    Newsroom

    Xamarin: Integration with Azure Functions

    Imagine a scenario wherein you need to quickly setup APIs or microservices for your POCs or small projects without the hassle of provisioning or managing of your infrastructures.
  • Dragon Model 3D ERNI26.03.2019.
    Newsroom

    New 3D printer for the enthusiasts in our Bratislava office

    At ERNI Slovakia, we have an established group of 3D enthusiasts. This week their new 3D printer arrived – bringing their passion to a new level. The machine came in parts so the printer had to be assembled. Michal Molčány and Miloš Karas tell us more about it.
  • Trade-offs 2019 ERNI22.03.2019.
    Newsroom

    5 trade-offs an enterprise software developer will have to deal with in 2019

    Software developers are facing a lot of trade-offs every day. This list is not meant to be complete. It is just a selection of interesting trade-offs we expect to pop-up during this year. Also, there is no simple answer to the trade-offs we discuss here. The correct answer is, as always, “it depends”.
  • Breeding fallow deers20.03.2019.
    Newsroom

    Adopted fallow deers at Bratislava ZOO

    The Marketing team of ERNI dedicated its internal “Hero” reward to the breeding of fallow deers at the ZOO in Bratislava.
  • Mobile World Congress 2019 - Outside view12.03.2019.
    Newsroom

    MWC19 – Minutes

    Last week we had the opportunity to participate in the MWC19 as Friends of the Swiss Pavilion.
  • Mobile Apps Advantages05.03.2019.
    Newsroom

    Apps have a lot of indispensable advantages in customer communication

    Let us talk about the advantages that apps bring to the customer interactions and communication.
  • at the glance 2019 ERNI04.03.2019.
    Newsroom

    New subpage with facts from the world of ERNI

    Did you know that this year we changed the format of our information about the facts and figures from the world of ERNI?
  • goodwill of design01.03.2019.
    Newsroom

    The goodwill of design

    The management consultancy McKinsey not only dedicated a study to this topic in 2018, but also developed the McKinsey Design Index (MDI) from it. This should quantify the business value of an appealing design.
  • 26.02.2019.
    Newsroom

    How it works at ERNI: Recruiting and hiring

    Each ERNI site has its own way of hiring and recruiting.
  • cell phone apps erni xamarin25.02.2019.
    Newsroom

    Profiling memory leaks in Xamarin Forms

    I remember one time I was working with a mobile project and it was already deployed to the client. It work out well from planning, development, testing to deployment but after a while the client was clamoring that the app is getting slower and slower when it was continuously being used up to the point where it will just crash suddenly. Well it is a given fact maybe not all but I’m guilty of it that we really love our code, so in that sense I was pointing the blame on maybe the device or something unusual that the client was doing with the Application. To make the long story short I’ve learned that Profilers are programmer’s best friend.
  • Mobile Apps Market Growth22.02.2019.
    Newsroom

    Rapid growth in worldwide app downloads

    The research company “App Annie” has revealed that worldwide app downloads have increased by 60 percent between 2015 and 2017 to a total of more than 175 billion annually – statistically this means almost two downloaded apps per inhabitant per month. No wonder, since the number of apps available on Google Play and the iOS App Store has now grown to more than six million.
  • Speaker from Theodora Foundation21.02.2019.
    Newsroom

    ERNI collaboration with Theodora foundation

    Last year 2018 ERNI Spain had the opportunity to start a collaboration with Theodora Foundation bringing smiles to the hospitalized children and their families.
  • Hans Hess President of Swissmem21.02.2019.
    Newsroom

    Opportunities and challenges of digitalisation in industry

    Since 2010, Hans Hess has been President of Swissmem, Switzerland’s largest industrial association, and Vice-President of economiesuisse, the Swiss Business Federation. For our .experience magazine, he contributed with his guest article on industry 4.0.
  • WMC Barcelona 201820.02.2019.
    Newsroom

    ERNI at the Barcelona Mobile World Congress 2019

    ERNI will be sponsoring and attending the Barcelona Mobile World Congress from 25-28 February organized by the GSMA within the Swiss Pavilion.
  • understanding design bloom erni19.02.2019.
    Newsroom

    Understanding design

    Apple founder Steve Jobs once said: “We made the buttons on the screen look so good you’ll want to lick them.” Jobs was known for placing a high value on the design of his products. But he wasn’t the first.
  • Pedco and ERNI partnership SAFe13.02.2019.
    Newsroom

    Partnership between ERNI and PEDCO

    PEDCO and ERNI will work together in fostering the growing numbers of Applied SAFe® customers; where PEDCO delivers its own product Applied SAFe® and ERNI with its 25 years of experience in Swiss Software Engineering delivers consulting, technology, delivery, and agile coaching in order to bring the Scaled Agile Framework to live at Applied SAFe® customers.
  • 12.02.2019.
    Newsroom

    Three main principles for enterprises wanting to go Agile

    As a more than 4-year-long SAFe® practitioner and trainer, I see that SAFe offers a lot of chances for our customers. Generally, in the long run, enterprises want and need to change the way they are working, but often do not know where to start.
  • How it works Outsourcing Shoring07.02.2019.
    Newsroom

    How it works at ERNI: Outsourcing and shoring

    Nearshoring as a way to focus on the core business and let a nearby location take care of your software development is around for some decade. However, the competition among nearshore locations in Europe has intensified and it is not an easy decision to pick the right one for your needs.
  • Process automation30.01.2019.
    Newsroom

    Techletter: What is RPA and what is it not?

    RPA is a buzzword of today. What can we imagine under the term and what definitely not? Let us explore the topic more deeply.
  • Cottolengo Filipino28.01.2019.
    Newsroom

    Cottolengo Outreach Program – Their own amazing world

    Last month our ERNI team in the Philippines experienced a memorable event. We all got united for one purpose and that is to extend our generosity and love. This year, instead of having the usual exchange gifts we came up with an idea to raise funds and give donations to our chosen charity. This program was called “Exchange your gifts to those in Need”.  
  • 25.01.2019.
    Newsroom

    Working Agile

    How-to: a simple introduction based on a workshop for the HR department.
  • transformation journey famous silos21.01.2019.
    Newsroom

    The infamous silos

    When we as ERNI visit a customer for the first time and they explain to us how the company works and what their structures and internal conflicts are, we typically recognise the three infamous silos.
  • Cross-platform Mobile Apps15.01.2019.
    Newsroom

    Cross-platform apps as the recommended solution

    The advantages of cross-platform apps are similar to those of a hybrid application. Eager to know why they are a recommended app solution? Read our article.
  • 3D Printing How it works10.01.2019.
    Newsroom

    How it works: 3D Printing

    In a nutshell, a physical object is created from a virtual design, i.e. the digital design is converted into a physical three-dimensional object by adding material layer by layer until the object is formed.
  • ERNI Slovakia donation christmas08.01.2019.
    Newsroom

    Donation to Slovak foster homes

    At ERNI Slovakia, we took the opportunity to make some dreams of kids in Slovak foster homes come true.
  • pollution measurement header07.01.2019.
    Newsroom

    Dust Measurement Network: Building an IoT Sensor Network

    How do you get from a great idea which you believe could have a significant impact to a tangible product? In a recent community project at ERNI, we put ourselves to this task. This is the second article out of a series of four, in which we explain how we built the IoT sensor network. Have a look at article one. The topic was presented at the Google Developers DevFest Switzerland 2018 conference. Watch the talk online.
  • Swiss Mobile Apps on the market18.12.2018.
    Newsroom

    Swiss applications have a hard time on the market

    Currently, according to Statista, WhatsApp is by far the leader among cross-platform apps (Android and iOS), followed by Facebook, 20 Minuten, Instagram and various weather apps.
  • Innovation from Switzerland13.12.2018.
    Newsroom

    Support from Switzerland

    Switzerland has been named the most innovative country in the world for eight consecutive years, according to the Global Innovation Index.
  • Step by step towards innovation06.12.2018.
    Newsroom

    Step by step towards innovation

    When we talk about innovation in general, we talk about three steps. Let us explore which ones.
  • Hack and Dive04.12.2018.
    Newsroom

    Hack & Dive in the Philippines

    On November 9, 2018, ERNI Philippines held its 1st Hack & Dive community activity.
  • Geoffrey Reyes, Professional Test Engineer27.11.2018.
    Newsroom

    Techletter: Cybersecurity is the next big thing

    Why is cybersecurity gaining importance for everyone and how can you eliminate risks? Read our newest Technology Post.
  • Mobile Apps Development Team Philippines26.11.2018.
    Newsroom

    The Mobile Team from the Philippines

    Another team portrait from the Our Teams series in awaiting you on the following lines.
  • Mobile Apps for your Business22.11.2018.
    Newsroom

    Apps – The most important channel of customer interaction

    Apps have become the most important channel for customer interaction for almost all industries and countless companies within just a few years. This is particularly true for so-called Mobile-First markets, where mobile devices are the No. 1 platform and many of their users have completely skipped the generation of desktop devices at the desk.
  • Different Lego Parts20.11.2018.
    Newsroom

    How well-established companies can embrace agile

    Recent rapid developments in business and technology require businesses to become faster, more global, more agile. 
  • pollution measurement ERNI13.11.2018.
    Newsroom

    Dust Measurement Network: Shaping the idea

    How do you get from a great idea which you believe could have a significant impact to a tangible product?
  • 13.11.2018.
    Newsroom

    Are PWA the future of apps?

    What attributes make Progressive Web Apps attractive? Explore our second article.
  • 5 Years at ERNI - Event in Barcelona06.11.2018.
    Newsroom

    5 Years at ERNI Event in Barcelona

    On October 15-16, 2018 we came together in beautiful city of Barcelona to celebrate the 5 Years @ERNI Event.
  • ERNI Spain Dream Team06.11.2018.
    Newsroom

    Our teams: The Dream Team from ERNI Spain

    We all have dreams and dreams are always good. Especially if you have a dream within your team.
  • Our Teams - Deadpool -We know we are awesome06.11.2018.
    Newsroom

    Our teams: Deadpool – “We know we are awesome”

    In our new series we present you our teams at ERNI. Get to know us better.
  • ERNI - GDG Progressive Web Apps Presentation02.11.2018.
    Newsroom

    What is a PWA?

    Progressive Web Apps: What does it mean particularly? Let’s explore it deeper in the article.
  • Michal Molčány, Kayaker and Professional .NET Developer26.10.2018.
    Newsroom

    ERNI Germany breaks new ground

    We have entered into a strategic partnership with QCENTRIS in order to grow faster in the German market.
  • Social Engagement Series - The Socially Responsible ERNI Hero Team12.10.2018.
    Newsroom

    Social engagement series: The socially responsible ERNI Hero Team

    Being socially responsible is a respectable act. Do you agree? We do.
  • Oscar Vaquero, Senior Consultant08.10.2018.
    Newsroom

    Agriculture 4.0: IoT – the new era

    What are the challenges of the modern agriculture? And what uses does IoT have in this field?
  • Hack and Hike Slovakia 2018 - Hiking28.09.2018.
    Newsroom

    Hack and Hike Slovakia 2018

    After a successful Hack and Hike event in Switzerland, another one took place in Slovakia this weekend. Twenty employees joined this event in order to spend some free time with their co-workers, enjoy the fresh mountain air and organise a few hack sessions.
  • ERNI at the Fintech Switzerland in Madrid24.09.2018.
    Newsroom

    ERNI at the Fintech Switzerland in Madrid

    Attending conferences is a door opener in every business area. In Madrid, The Swiss Business Hub Spain as representative of the Swiss trade and investment promotion agency Switzerland Global Enterprise (S-GE) and part of the Embassy of Switzerland in Spain organized the Swiss fintech conference and B2B event FINTECH SWITZERLAND.
  • Nuria Lluch, Barcelona Lab Manager12.09.2018.
    Newsroom

    Three ERNIans in the U.S.A.

    Since the beginning of 2018, starting with our market entry to the U.S., a group of three ERNI experts has been performing work for our medical technology customer in the States. What is it like to be working in the U.S.A. and how are our colleagues doing overseas? We talked to one of them – Nuria Lluch, our Barcelona Lab Manager about the experiences.
  • The Swiss Hack and Hike - Landscape12.09.2018.
    Newsroom

    The Swiss Hack and Hike

    At the end of every summer, the developers meet and spend a weekend together surrounded by nature. This activity, called Hack & Hike, brings an opportunity to escape from daily routine, clear the minds by hiking and come up with new ideas during the hack sessions.
  • Madagascar Landscape22.08.2018.
    Newsroom

    Social Engagement Series: Talking digital in Madagascar

    Previously, we already talked about the healthy lifestyle pages of our colleagues from ERNI Philippines and the engagement of our Slovak colleagues for the Capital city of Bratislava. Today, Stefan from ERNI Switzerland tells about his support of a non-governmental organisation in Madagascar.
  • HTML, CSS, JavaScript and JQuery Books20.08.2018.
    Newsroom

    My favourite programming language: Why I don’t have the ONE favourite programming language

    Oliver Gramberg from ERNI Switzerland closes up our series My favourite programming language. Read why he has NOT ONE single programming language.
  • Nina Duchonová, Professional Software Developer, ERNI Slovakia14.08.2018.
    Newsroom

    Straightforward – Python

    What is so special about Python? Nina Duchoňová, Professional Software Developer tells us about her affinity for this programming language.
  • Zuzana Drábová, Professional IT Manager10.08.2018.
    Newsroom

    The challenge of Camino de Santiago

    As on the job, also in life, walking challenging ways pushes your inner strength forward.
  • John Ernest Ramos01.08.2018.
    Newsroom

    An app for a healthy lifestyle

    Read a success story of our colleague John who motivated also his colleagues to lead a healthy lifestyle and get back on the track
  • ERNI Slovakia Team - Volunteering for our city01.08.2018.
    Newsroom

    ERNI Slovakia Team: Volunteering for “our city”

    Since several years, at ERNI Slovakia our team has been volunteering within the initiave “Naše Mesto” making life in Bratislava more enjoyable.
  • Innovation in Medical Technology30.07.2018.
    Newsroom

    Innovation in Medical Technology

    The majority of challenges that businesses in the medtech industry have to face are related to innovation and market access, says a study carried out by the Swiss Medical Technology Institute.
  • The Internet of Medical Things24.07.2018.
    Newsroom

    The Internet of Medical Things

    Why it matters and what to do about it
  • Andreas Gisler, CEO of the ERNI Group24.07.2018.
    Newsroom

    Personnel changes at ERNI

    Andreas Gisler becomes CEO of the ERNI Group with immediate effect. He also remains Delegate of the Board of Directors of ERNI Global and thus responsible for the operational management of the company with the three divisions ERNI, SEI and People Business.
  • Discovering Azure Workshop23.07.2018.
    Newsroom

    Discovering Azure Workshop

    On 19 July 2018, the ERNI Philippines Community Core Group held its 4th External Community Event in the company premises.
  • How To Embrace Change23.07.2018.
    Newsroom

    How to embrace change

    Do an experiment, approach your friends or colleagues and ask: “How do you think you will have changed in the next 15-20 years?”
  • How DIY Medical Testing Could Save Your Life17.07.2018.
    Newsroom

    How DIY Medical Testing Could Save Your Life

    During a TEDx talk in Calgary, Alberta last summer, MIT doctoral student Steven Keating explained how curiosity about his brain led him to volunteer to have it scanned for a study.
  • Anton Beskov, Senior Software Developer12.07.2018.
    Newsroom

    Techletter: Moving an existing Java Web application to the Cloud Foundry Platform

    “It’s easy to create and run a new cloud-native application, but migrating an existing Java Web application to the cloud might be quite complex: most applications have to be changed in a way that makes them incompatible with the old environment and thus additional risks arise. In this article, we’re sharing our experience with the migration of large Web applications, including some advice and best practices.”
  • Inside ERNI Lab 409.07.2018.
    Newsroom

    Inside ERNI Lab

    Testing the latest medical technologies
  • Ready to take a step towards data06.07.2018.
    Newsroom

    Ready to take a step towards data?

    An overview of the skills needed for both small- and large-scale projects
  • Tudor Ciotlos, Senior Software Developer at ERNI Romania06.07.2018.
    Newsroom

    My favorite programming language: TypeScript

    As a developer who regularly uses multiple programming languages, it’s always hard to single one out as the “favourite programming language”.
  • Agile: The Answer to Working with Data06.07.2018.
    Newsroom

    Agile: The Answer to Working with Data

    Agile methods and sprint-based development make our teams more effective.
  • Successful Rapid Development through Agile Infographic06.07.2018.
    Newsroom

    Successful Rapid Development through Agile

    Agile is a methodology that can be used to launch new software projects efficiently and rapidly.
  • An External Scrum Master06.07.2018.
    Newsroom

    Two cases of Agile and Scrum

    Read these two case studies about external scrum master and nearshored scrum team
  • Engineer meets creative thinker - ERNI cooperates with design agency Bloom Identity03.07.2018.
    Newsroom

    Engineer meets creative thinker: ERNI cooperates with design agency Bloom Identity

    With Bloom Identity, the ERNI Group is teaming up with an agency that specialises in brand consulting and thus enhancing the customer experience services it offers to customers.
  • Learning Culture pays off for Employers and Employees01.07.2018.
    Newsroom

    Why a thriving learning culture pays off for employers and employees

    An active learning culture is a crucial part of a modern organisation that wants to attract and nurture the best talent. Fostering communities is a great way to achieve this.
  • Marek Linka, Senior Software Developer26.06.2018.
    Newsroom

    Operationalizing machine learning models 3/3 – Production-grade, scalable deployment

    The previous two articles in this series introduced the Azure Machine Learning Workbench and walked us through the process of wrapping our ML model in a self-contained Docker image. Now is the time to take this image and deploy it into a production-grade environment.
  • Thomas Ong20.06.2018.
    Newsroom

    C# is simple. C# is object-oriented. C# is powerful and flexible.

    My favorite programming language
  • Shared vision for changes16.06.2018.
    Newsroom

    How to create a shared vision for changes – The Power of visualisation

    In change initiatives one key success factor is to create a shared view on what the future will look like. Visualising of the envisioned target state is a powerful method for guiding the change team on their way through the change.
  • Marek Linka, Senior Software Developer31.05.2018.
    Newsroom

    Operationalizing machine learning models 2/3 – From model to service

    The first part of this series of articles introduced the Azure Machine Learning Workbench and showed us how to setup the resources required to operationalize our machine learning models In this article, we will see how to get our model into Azure and run it locally for testing purposes. Let’s get to it!
  • Marek Linka, Senior Software Developer29.05.2018.
    Newsroom

    Operationalizing machine learning models 1/3 – Azure ML Workbench

    Machine learning and AI are everywhere. More and more people are discovering the advantages that learning algorithms can provide – decreasing costs, improving efficiency, eliminating menial tasks, etc. Knowing at least a little bit about how these algorithms work is quickly becoming a necessity. Read it now.
  • Isabelle Rüthemann23.05.2018.
    Newsroom

    Work in the innovation sphere gives me the chance to develop myself

    Interns at ERNI series: Isabelle Rüthemann
  • Interns at ERNI series: Adrian Chiorean22.05.2018.
    Newsroom

    I have greatly improved my vision and understanding of programming

    Interns at ERNI series: Adrian Chiorean
  • ERNI and IoT Projects15.05.2018.
    Newsroom

    ERNI and IoT projects

    Many of our customers’ data projects are subsets of larger Internet of Things initiatives. Although the IoT produces data, the scope of the IoT projects is usually more extensive and covers a broader pool of activities. This illustration describes/shows the ERNI collaboration model for IoT projects.
  • Interns at ERNI series: Viera Maronová11.05.2018.
    Newsroom

    ERNI gives us a space for our own dreams

    Interns at ERNI series: Viera Maronová
  • Nuria Lluch, Basketball player and Test & Laboratory Manager09.05.2018.
    Newsroom

    How we have changed our visual language

    A cyclist full of energy, a passionate painter, a handball player jumping in the air with a clear goal to score: Anyone who is currently looking at the external presentation of ERNI can see a clear move.
  • Stricter regulations on personal data even outside of health records GDPR03.05.2018.
    Newsroom

    Stricter regulations on personal data even outside of health records GDPR

    The upcoming EU General Data Protection Regulation (GDPR) is the most important change in data privacy regulation in 20 years.
  • Interns at ERNI series: Nikoleta Žišková27.04.2018.
    Newsroom

    Why it is important to work during your studies

    Interns at ERNI series: Nikoleta Žišková
  • How ERNI Uses Data to Generate Value24.04.2018.
    Newsroom

    How ERNI Uses Data to Generate Value

    Five practical case studies
  • Data Science Projects - How to Manage Stakeholders19.04.2018.
    Newsroom

    Data Science Projects: How to Manage Stakeholders

    The second part of our Managing Stakeholders article evolves around further phases of data projects and what stakeholders to involve.
  • ERNI Development Day in Bern17.04.2018.
    Newsroom

    ERNI Development Day in Bern

    Developing ourselves with many interesting sessions.
  • ERNI Hero - Darina Stipula16.04.2018.
    Newsroom

    ERNI Hero: Darina Stipula

    We are all working together at ERNI, but not all of us can see each other daily. Some of us have the opportunity to work closer with other teams and know each other better – long distance is not a limiting factor anymore.
  • Geneva Community Workshop16.04.2018.
    Newsroom

    Geneva Community Workshop

    Why is culture an important element of our company? Check the video from our latest workshop in ERNI Geneva and learn more.
  • 7 under 30 - Radu Sipos12.04.2018.
    Newsroom

    7 under 30: Radu Sipos

    Radu has a bachelor degree in Robotics.
  • Shredded CDs11.04.2018.
    Newsroom

    Missed security pitfalls

    It is not just securing the transmission and storage of data against hacks that is important. Here are some easily missed security pitfalls.
  • 7 under 30 - Veronika Piknova09.04.2018.
    Newsroom

    7 under 30: Veronika Piknova

    Get to know some of our young and successful ERNIans!
  • Rastislav Novotný, Principal08.04.2018.
    Newsroom

    VSTS/TFS automated build agent provisioning

    Continuous Integration / Delivery is bread and butter of every agile development team nowadays. Visual Studio Team Services or Team Foundation Server is one of many tools providing this feature. Although I will use VSTS/TFS in this article same process can be used with any other CI/CD tool.
  • Managing Stakeholders in Data Science06.04.2018.
    Newsroom

    Managing Stakeholders in Data Science

    Innovation can be hard to implement if there is no common understanding about it between the organisation’s stakeholders (colleagues, partners and leadership). For a project manager, stakeholder management is not only important but it can also often make or break the project.
  • Joseph Boro04.04.2018.
    Newsroom

    Leaner Mobile DevOps Implementation Using Visual Studio App Center

    DevOps is an important facet of mobile app development that is now a requirement for a successful app life cycle. When building and delivering mobile apps nowadays, one has to consider setting up a lot of different things.
  • 7 under 30 - Marek Linka22.03.2018.
    Newsroom

    7 under 30: Marek Linka

    In our 7 under 30 series, we are introducing you our young and successful ERNIans!
  • Veronika Piknova16.03.2018.
    Newsroom

    The new ERNI Website

    We have worked hard on the implementation of our website. The result is a page with a new design, new content and simplified navigation. It fits into the new overall appearance of the ERNI Group.
  • Nicolas Friberg16.03.2018.
    Newsroom

    Major methodology trends in software development for 2018

    An integral part of consultancy and software development in a VUCA world – meaning volatile, uncertain, complex and ambiguine – is focusing on making feedback loops shorter and more efficient. The agile mindset is spreading within organisations – from agile software development to business or corporate. Here are the 5 major methodology trends we see for 2018.
  • ERNI Hero - Ruel Dognidon15.03.2018.
    Newsroom

    ERNI Hero: Ruel Dognidon

    Are you sitting next to a Hero? Has your team member positive impact on personal and company development or social engagement?
  • High Performance Teams in an Agile Environment15.03.2018.
    Newsroom

    High-performance teams in an agile environment

    With the recent status of digital transformation, besides the technical skills of teams, soft skills for fostering collaboration are gaining even more importance.
  • 7 under 30 - Anastasiia Shelukhina14.03.2018.
    Newsroom

    7 under 30: Anastasiia Shelukhina

    Get to know some of our young and successful ERNIans!
  • Miloš Karas, Climber and Professional Test Engineer13.03.2018.
    Newsroom

    Raising efficiency with user adoption

    Applying “Business-centric user experience” is decisive for whether an application will be used by the end users or will fail.
  • On what should we focus when we implement microservices13.03.2018.
  • Marco Stöckli12.03.2018.
    Newsroom

    From business idea to business case

    Existing business models must be reviewed during digital transformation, with new business ideas developed, sorted and evaluated.
  • ERNI Hero - Dieter Niklaus07.03.2018.
    Newsroom

    ERNI Hero: Dieter Niklaus

    This is the first time at ERNI, that our employees had the chance to nominate any of their colleagues for their positive approach or social engagement.
  • 7 under 30 - Laura Gallardo05.03.2018.
    Newsroom

    7 under 30: Laura Gallardo

    Laura is 28, married and living in a small village close to the mountain called Begues, Barcelona.
  • Taking on Full Responsibility for Innovative Applications02.03.2018.
    Newsroom

    Taking on full responsibility for innovative applications

    “DevOps as a Service” ensures that a solution with a directly measurable benefit is found.
  • 7 under 30 - Kris Lenard Ocampo Fajardo02.03.2018.
    Newsroom

    7 under 30: Kris Lenard Ocampo Fajardo

    Get to know some of our young and successful ERNIans!
  • Toni de la Torre - Biker and Senior .NET Developer01.03.2018.
    Newsroom

    Leading the innovation process towards sustainability

    “Innovation as a Service” delivers customised methods for the entire process from creating ideas to implementation of a solution with measurable added value.
  • Continuous IT Governance Driving Transformation27.02.2018.
    Newsroom

    Continuous IT governance driving transformation

    With the adoption of smart technologies for business innovation, it is crucial for IT governance to be a process of continuous improvement.
  • Tatiana Abrhanová, Runner and Senior Marketing Specialist25.02.2018.
    Newsroom

    Never start from scratch again

    The beauty of digital solutions lies in their ability to adapt to customer specifics. The underlying technology framework does not have to be recreated each time.
  • 7 under 30 - Enrique S. Cavestany25.02.2018.
    Newsroom

    7 under 30: Enrique S. Cavestany

    Enrique joined ERNI shortly after graduation and ever since has devoted himself to his responsibilities of quality assurance.
  • Vendors that help with digital transformation24.02.2018.
    Newsroom

    Vendors that help with digital transformation

    Vendors can no longer only rely on the monitoring and management of IT infrastructures to keep pace with market needs.
> Load more

ERNI Schweiz

Casinoplatz 2

3011 Bern

Phone: +41 58 268 12 00

Email: [email protected]

better ask ERNI

ERNI Suisse

Bâtiment L

Route des Acacias 43

1227 Geneva

Phone: +41 58 268 11 03

Email: [email protected]

better ask ERNI

ERNI Suisse

Voie du Chariot 3

1003 Lausanne

Phone: +41 58 268 11 03

Email: [email protected]

better ask ERNI

ERNI Schweiz

Brünigstrasse 18

6005 Lucerne

Phone: +41 58 268 12 00

Email: [email protected]

better ask ERNI

ERNI Schweiz

Löwenstrasse 11

8001 Zürich

Phone: +41 58 268 12 00

Email: [email protected]

better ask ERNI

ERNI Deutschland

Trakehner Str. 7-9

60487 Frankfurt am Main

Phone: +49 697 953 28 01

Email: [email protected]

better ask ERNI

ERNI Deutschland

Design Offices München – Arnulfpark

Luise-Ullrich-Str. 20

80636 München

Phone: +49 162 334 77 30

Email: [email protected]

better ask ERNI

ERNI Slovakia

Ševčenkova 34

851 01 Bratislava

Phone: +421 2 32 55 37 37

Email: [email protected]

better ask ERNI

ERNI España

Edificio El Triangle

Plaça Catalunya 1-4, 3º planta, Módulo A y B

08002 Barcelona

Phone: +34 93 667 77 76

Email: [email protected]

better ask ERNI

ERNI España

Carrer Pallars, 208, Bajos

08005 Barcelona

Phone: +34 93 667 77 76

Email: [email protected]

better ask ERNI

ERNI España

Calle de Alfonso XII 62

Oficina 3101

28014 Madrid

Phone: +34 901 848 787

Email: [email protected]

better ask ERNI

ERNI España

Sant Cugat ERNI Office

Plaça Xavier Cugat, 2 EDIF B Planta Baja

08174 Sant Cugat del Vallès

Phone: +34 93 667 77 76

Email: [email protected]

better ask ERNI

ERNI Romania

Calea Dorobantilor no. 98-100

3rd floor, in Olimpia Business Center

400609 Cluj-Napoca

Phone: +40 744 319 228

Email: [email protected]

better ask ERNI

ERNI Singapore

7 Straits View

Marina One East Tower #05-01

Singapore 018936

Phone: +65 9161 9863

Email: [email protected]

better ask ERNI

ERNI Philippines

9th Floor, 500 Shaw Zentrum Building

500 Shaw Boulevard

Mandaluyong City, Philippines 1555

Phone: +63 2 531 59 82

Email: [email protected]

better ask ERNI