Long read: How being ‘selfish’ serves the collective interest in the end

Reading Time: 8 minutes

How a common goal can be reached despite clashing individual interests

  • The following white-paper is part of Jochem Hummel research on ‘Collaboration and innovation between heterogeneous actors’. Jochem will be defending his research on June 5th, 13:45, in the Aula of the Vrije Universiteit
  • Right before his defense, there will be a symposium on collaboration and innovation between heterogeneous actors, organized by the Amsterdam Business Research Institute (ABRI), KIN Center of Digital Innovation (School of Business and Economics), and Warwick Business School.
  • 10 min read

They all wanted the same thing back in 2011, a leading computing cloud infrastructure and platform for European science that would enable them to do and share research at the highest possible level. Quickly became clear that the initiators, the three largest scientific research organisations in Europe, expected different things from the cloud platform. Also, the several commercial parties that offered cloud computing platforms at the time were not offering what the academic world was looking for. Despite all these different backgrounds and goals, a collaboration amongst these actors was initiated to develop a leading computing cloud infrastructure for European science: Helix Nebula.

In this eight year longitudinal research of Jochem Hummel, Hans Berends and Philipp Tuertscher from the KIN Centre for Digital Innovation at the Vrije Universiteit Amsterdam, we do not focus on the specific – technical – outcome of the cloud computing infrastructure. It is far more interesting to look at it from a management perspective; how to manage this complex collaborative development process. We were interested in understanding how very different worlds – big business and big science – successfully worked together to develop a collective product although the individual interests were not the same.

Other organisations, facing similar complex processes and partnerships as part of their digital transformation can learn from this revelatory case and the process model that has been developed now by Jochem, Hans and Philipp explaining the bottom-up development of common resources.

The actors in the Helix Nebula playfield

In 2011 CERN, the European Organisation for Nuclear Research in Geneva, Switzerland, known for its 27-kilometre particle accelerator, inventor of the World Wide Web, and as one of the three largest science organisations in Europe, was aware of its future need for cloud computing resources to perform high-energy physics research. Together with the other big science organisationsin Europe, EMBL, the European Molecular Biology Laboratory in Heidelberg, Germany and ESA, the European Space Agency in Frascati, Italy, CERN initiated a collaboration to develop a leading cloud computing infrastructure for science in Europe. Together, CERN, EMBL and ESA attracted big business organisationsto join Helix Nebula: including Atos, CloudSigma and T-Systems.

A range of other organisations also joined Helix Nebula, like software development companies and public e-infrastructure providers. In total between 2011 and 2018 70 of these parties collaborated in Helix Nebula.

Last but not least, the European Commission was an important stakeholder in this program as they were keen to develop a cloud computing infrastructure for science within and for the European Union – as evidenced by the 2 billion Horizon 2020 funding allocated for the development of a European Open Science Cloud.

Bottom-up development

Starting off a project like Helix Nebula the logical first step is to identify the individual interests of all parties involved and to define where there is common ground in these individual interests.

Where CERN primarily needed extensive cloud computing power at very specific moments for example, ESA wanted to store and share publicly extremely large images of the earth and space and EMBL needed memory-based computing power, amongst other things. Also, the big business organisations each offered their specific knowledge, experience, expertise, models and forms of contracting that would serve one or more interests of the scientific institutions, but not necessarily the other(s). So, there was not only a gap in how big science and big business looked at things, but there were also different approaches, views and opinions within the entities.

A situation like this could easily turn into a deadlock where all parties could conclude that developing a common leading cloud computing infrastructure for European science will not be feasible as the individual interests do not correspond enough. Surprisingly, the parties involved in Helix Nebula decided to work together to overcome this deadlock by developing so called common resources (resources with shared ownership). However, when there were so many different interests at stake how do you develop something of shared ownership? Helix Nebula showed this is possible by splitting up to investigate several suggested options and sub-options in so-called micro-alliances.

The power of micro-alliances

Helix Nebula started off as a project where all actors were at the same level playing field, all agreed there was a need for a common technical cloud interface and all agreed to explore different options. But over time it became clear that some parties were pushing more towards a specific option. Or that none of the options was completely satisfactory to fill a given gap for example. How to solve this? How to get consensus? How do you decide what will be the right choice?

So where individual interests are clear, how do you decide which out of these interests can become a legitimate option for a common resource? Which one is more dominant over another? Or which option is worth investing more time in? Or how long do you continue investigating an option to see if a suggested option could work in the end or not? To whom an option is positive, neutral or a threat? And most importantly, who is leading the discussion and how is decided what is the best solution?

Micro-alliances are flexible structures where representatives of organisations work together to investigate options. Micro-alliances are open structures, and in Helix Nebula organisations would typically take part in every micro-alliance of their interest and choice in order to explore multiple common resource options that were defined over time. The idea was that this way every option could be developed into detail by a motivated team.

Not every player was necessarily represented in every micro-alliance. For developing an option where one player, say CERN, would have much interest in, you could see that its representatives would actively participate and take decisions in the micro-alliance. CERN had a less dominant role in micro-alliances where they did not have much interest (being an endorsing actor in that case). In a collaboration like Helix Nebula with organisations spread across Europe, micro-alliances typically met in workshops, meetings, airports, conference calls and through mail.

This might come across as a lengthy and time-consuming process, but interestingly enough it saved time, enhanced the quality of the outcome and it made sure progress was made. In a micro-alliance there can be absolute focus on developing details and no distraction for factors that were not relevant to that task.

Reconfiguring common resource options

In the course of time, the micro-alliances in Helix Nebula each developed options for the common technical cloud interface that differed a lot. Three options were defined, each supported by a different set of organisations or micro-alliances. In this phase of the process, a decision that was supported by all parties could not be taken. There were too many differences and dependencies between the different options. You can compare this to some extent with the current situation of standards for charging electrical vehicles, where there are competing plug options.

The academic world and the business world had to overcome their different natural behaviours that come with their backgrounds and identity. Where big businessis more about keeping control, working towards a fixed outcome, working according to a plan and if needed settling for a compromise; big scienceworks more along the line of working with what arises, adapting to new findings and changing the plan accordingly as they ‘simply’ want the best solution possible at any given time.

Interestingly, we saw that as all actors were acting responsibly and respectfully to each other, holding on to individual interests served the collective interest in the end. What seemed a purely individual interest at first, revealed later to be a collective one. Not all parties involved were always aware of that at the same time. Not surprising at all, the more money was at stake the more individual the actors operated in their own interest.

Breakthrough: splitting the technical and non-technical aspects of the infrastructure

In Helix Nebula, the people who represented the scientific institutions were the finest academics. They were primarily interested in the technical aspects of the cloud computing infrastructure they wanted to build for European science. That was leading in every discussion. Non-technical aspects were also discussed on the side in different micro-alliances: How are we going to pay for cloud in Helix Nebula? And who pays what? What is the right governance model? Which contract model will be chosen? Which common services are we going to have? Where will it be based? What are membership guidelines? These non-technological questions could be answered more collectively than individually as it was the case discussing the technical options. Splitting the technical aspects of the cloud computing infrastructure from the non-technical aspects, was a breakthrough in the discussions. It brought less distraction, and therefore more focus in discussing the several options within the micro-alliances. It is important to state that in the area where there was no competitiveness between the parties, parties expressed their distinctiveness in other areas.

In discussing the non-technological aspects of the infrastructure there were still different options to be considered of course. These options were all discussed in several micro-alliances where there were continuously evolving and reconfiguring options and micro-alliances as discussed earlier.

No compromise, no strings attached

Cloud computing is more flexible, efficient and up to date than buying expensive hardware from commercial parties. Scientists want to have access to different types of computing power or data storage at different moments in time for different scientific purposes. Scientists do not compromise, they want the latest and the best solution for their research. They could not care less if there are constraints in the commercial world. They want to keep all options open at all times and have the freedom to choose the best answer for their scientific question. Whereas the commercial world often seeks for a compromise in order not to risk to lose their market share to a competitor.

The academic world knew in 2011 that they needed the experience and knowledge and building power of the commercial world. Just as the business organisations needed the academic world to detect opportunities and possibilities they could not have uncovered without the relentless dynamics of the scientific world. The level of knowledge and expertise they have taken home from Helix Nebula has brought them large new opportunities in the commercial world.

It is fair to say that cloud computing has become a commodity nowadays. The European cloud computing system is in effect, but as one can expect from the Helix Nebula approach things have evolved from here in the meantime. CERN and other stakeholders have explored procurement contracts to flexibly buy and procure cloud computing services and in 2019 will start developing together with big businesses innovative ways to transfer large amounts of data across different commercial clouds and users (CERN has more than 300 petabytes of data archived).  

Out-of-the-box 10.0

Scientists are used to the idea that they can solve complex questions internally, within the scientific world. Commercial organisations on the other hand, think in revenue, shareholder value and often short term benefits. In Helix Nebula the scientific world understood that they needed big business organisations to build the leading cloud computing infrastructure together with them. And the commercial organisations saw step by step that the big science organisations could open up a new world of possibilities for them. Both worlds learned in the course of the seven years that letting the fences down, being less protectionist and really working together, would lead them to that common goal. And would be serving their individual interests even better in the end.

Also, by acknowledging the fact that there simply was no alternative for the individual interests progress was made in the process. The base was that every stakeholder should be able to profit from Helix Nebula. This common ground: the importance of the individual interest of every party was crucial in making steps forward in the process. It is this vision and patience that in the end made the development of Helix Nebula possible. By being ‘selfish’ every single stakeholder served the collective interest in the end.

“This research is supported by Netherlands Organisation for Scientific Research, project number 409-13-212”

Author: Jochem Hummel