The Weather Enterprise: A Global Public-Private Partnership

The weather enterprise is a well-established and successful global public-private partnership in which both sectors share common goals. There are new opportunities emerging to develop this partnership further that will enable the whole enterprise to grow and produce more accurate and reliable weather forecasts¹. The urgency to do this comes from the need to be even more effective in saving lives and protecting infrastructure because of vulnerability to weather hazards in a changing climate.

The scientific and technological success story

The development of weather forecasting is a scientific and technological success story. In what amounts to a scientific revolution² – albeit a quiet one taking many decades of progressive innovations – by 2012 it was possible to predict as extreme and unusual a hurricane as Sandy with around one week of advance warning.

Yet today, and into the future, people and infrastructure are increasingly vulnerable to weather hazards because of population increase, where people live and climate change. Therefore, the requirements society, business and governments have for accurate and reliable weather forecasts are growing rapidly.

The public-private global weather enterprise is rising to this challenge but within a context that is radically different from the one that has pertained thus far. New approaches are required to grasp opportunities and to deliver what is needed. 

How has the quiet revolution happened?

Three crucial ingredients came together over the last half century: Advances in weather science, including in modelling the global Earth system, innovation in observing the atmosphere, oceans and land surface, and a revolution in computing. Without all three, it would have been impossible to contemplate modern weather forecasting. When Vilhelm Bjerknes dreamed in 1904 of weather prediction using the laws of physics, it could not be realized³ but today, through both public and private contributions, the necessary ingredients have been brought together to make his dream reality.

Whilst most weather science has come from academia and research institutes, including those within national meteorological and hydrological services (NMHSs), innovation in observing and computing has had major contributions from the private sector. Today, some of the world’s largest companies at the heart of the global economy, for example in the space and computing industry, contribute to weather infrastructure. It is clear that the weather enterprise is a global public-private partnership.

It is crucial to recognize that this revolution, and so the public-private partnership, has required a fundamentally global approach as atmospheric circulation means that weather in one location is determined by prior events all over the world.

What of the future?

The need for more science, observations and computing power is enduring and continues to be the way forward to improve weather forecasts. However, the context for the weather enterprise is evolving rapidly today as exponential improvements in technology, including those driven by other industries, are creating exceptional opportunities to deliver even higher quality weather forecasts.

A key challenge is to mobilize sufficient human ingenuity to bring about innovation for the creation of more science, observations and computing. In turn, this innovation requires that sufficient financial resources be brought to bear. In the public sector, the availability of funds from governments is under heavy pressure. However, the private sector is undergoing significant development and growth.

What can the private sector offer?

The private sector is efficient in raising and deploying private (venture) capital particularly for high-tech developments in measurement and computing/data technology. If providing a data service, the private sector would also enable a transfer of risk, such as that associated with building, launching and operating satellites, from the public sector. Using its resources, the private sector could also assist in technology transfer to developing countries, for example, through World Bank funding.

In general, the private sector is recognized as capable and efficient in operationalizing innovation that has arisen from public sector investment in research and development – many governments invest in science in large part for this purpose. Many small and large companies already add value to public numerical weather prediction (NWP) data and disseminate weather forecasts widely.⁴

Recent developments in the private sector

Two examples are illustrative of how the weather enterprise context is changing. The first is that the development of small satellites – CubeSat⁵ – means that the cost of launching some new instruments into space for Earth observation is relatively low and also that the rapidity and number of new launches can be greatly increased. These facts mean that new small-to-medium sized companies are coming into existence utilizing private capital to launch satellites for meteorology – this is new and presents real opportunities.

These companies are interested in providing a data service – they see themselves in the data business as opposed to purely space hardware. This is also relatively new for the weather enterprise as in the past the cost and risk of launching and operating satellites was borne within the public sector. This requires new business models to be devised to ensure data availability; this would not imply more funding being made available but rather would require a re-structuring of how funding is allocated and risk apportioned. There are other companies which, building on the work originally carried out by some NMHSs, have developed instrument payloads for mounting on commercial aircraft that as a by-product enable a channel for aircraft-to-ground communications.

The second example relates to the countless developments and innovations in computing. For example, the growth of cloud computing or more generally distributed and remote computing capability has enabled companies and others to purchase computer cycles without the need to maintain their own supercomputing facilities. Then, there is the development of next generation computer chips that enable lower power consumption and enhanced performance. As NWP codes are not ideally structured for such next generation architectures, a much closer interaction between model developers and hardware/software vendors is needed if new technology is to be exploited. Typical procurement cycles by the major NWP centres are much slower - even ponderous - compared to the rapid developments in the computer industry.  Another innovation is the growth of location-specific data, often generated by data analytics bringing diverse data sources together to stream them to mobile phones and tablets. As weather forecasts are the most popular applications on such platforms, private data companies see the potential to bundle many other data with the weather forecasts, thereby ensuring that weather forecasts get into the hands of those who most need them.

A consequence of these developments is that some companies are recognizing that it is within their reach, and of significant commercial interest, to perform operational global and regional NWP themselves and to use private observational data services. The market for tailored forecast data of relevance for a wide range of weather-dependent business sectors is expanding and providing new customers for the private sector.

Roles for both

What does this evolution and growth of the private sector component mean for the weather enterprise? Inevitably, because the private sector is becoming much more involved in nearly all elements of the pipeline that goes from observations to tailored weather products (see Figure at right), the respective roles of the public and private sectors are evolving. Secondly, the weather enterprise is growing because of scientific and technological innovation, the need to rise to the challenge of producing better weather forecasts, and the growth of the private sector.

This is good news. It means that there is a strong incentive for both the public and private sectors to work together in a more pervasive and sustainable partnership. Both sectors can benefit in these circumstances; the weather enterprise is not a zero-sum game.

But growing misunderstandings and even mistrust about the respective roles of the public and private sector are becoming stumbling blocks to further progress of the weather enterprise partnership. There needs to be a much greater engagement between the two sectors to dispel perceived obstacles and to change the underlying mindset. The mission statements of most organizations involved in the weather enterprise, whether they be from the public or private sectors, are very similar and focus on the need to enhance the ability of weather forecasts to save lives and protect property. Mistrust arises because of a lack of knowledge and clarity about the respective roles of the two sectors and regarding how they can best work together.

Evolution of the public sector role

An indication of the thinking taking place within NMHSs on these issues is provided by the recently approved “2016–2025 Strategy of the European NMHSs”, which has been agreed by 35 European NMHSs. This strategy states that: “In response to the anticipated growth of the private meteorology sector, the distinct roles of the European NMHSs with respect to data collection, model development, research, warnings and alerts need to be established, while at the same time collaboration with the private sector is stimulated.”

A key role for the public sector is in the long-term research and development needed to improve understanding of weather and in the use of that knowledge within NWP modelling codes. Private companies recognize that the weather enterprise has been built, and must continue to be built, on public sector investment in both the backbone of global observations and in basic research and development. However, the private sector can contribute in these areas by, for example, funding specific research projects.

For this public sector responsibility to be carried out, taxpayers, that is to say governments, have to be persuaded to provide funding. The private sector needs to step up to provide the ammunition to win that argument. Many governments understand that at the heart of a modern knowledge-based economy is public investment in science and technology. It is an investment that pays back, amongst others, in the economic benefits that accrue from jobs and wealth creation due to private sector exploitation of this innovation and open access to public data. This is evidently already the case in the weather enterprise – a very good news story to tell.

Some in the public sector are worried about the emergence of observational data services in the private sector and the possibility that this might cause a breakdown in the current global arrangements whereby such data, paid for by national public investments, are shared free-of-charge via the WMO World Weather Watch. But with the risks come real opportunities for many more observations to be made. Private sector companies have indicated their support for WMO Resolution 40 and commitment to demonstrating the quality of their product as well as a desire to engage in a constructive way. To mitigate these risks, constructive and active engagement is needed between public and private sectors.

The weather forecasting role

Another area that needs more clarity is in the provision of information from operational NWP. The vast information stream from models can be used for the public weather forecast as well as for the tailored information needed for specific users. Public weather forecasts are designed to have wide utility, including to provide early warning of impending severe and hazardous weather. They are usually constructed from standardized sub-sets of the data output from NWP models. The tailored information needed by specific users can be constructed from these standardized data outputs using a variety of tools including post-processing/calibration, forecaster interpretation, and other value-adding techniques. But they can also be created by using non-standard data outputs to produce specific “forecasts” for particular uses/users.

NMHSs view it as crucial that they remain the single authoritative voice within their countries to warn the public regarding hazardous weather and for national security purposes. On the other hand, it is becoming clear that both NMHS and private companies are capable of creating operational global and regional NWP forecasts in-house. Private companies can and do tailor the information from NWP for a variety of business (and indeed public) customers. Private companies that run operational NWP models themselves would be able to produce both types of tailored information mentioned above. And for countries that lack the basic infrastructure there is the potential for non-national organizations, including private companies, to provide the public weather forecast also.

Complementarity

A market differentiation defining complementary roles may, therefore, be possible and desirable. Where possible, NMHSs would continue to provide the public and national contingency services with warnings and the private sector companies would provide business customers with tailored products. Again, constructive dialogue would seem essential to clarify these respective roles to at minimum prevent any confusion arising from multiple sources of weather forecasts.

However, it has to be recognized that duplication already exists regarding public weather forecasts as witnessed by the plethora of weather apps – the distinction between a forecast and a warning is perhaps lost on the public. It may be time for the WMO to consider a quality-assurance approach to inform the public of the inherent quality of the underlying forecast data in the various apps.

Regional diversity

The global framework of the weather enterprise belies large regional variations. Levels of public national investment in the weather enterprise are dramatically different between countries as is the interplay between the public and private sectors. Even for countries with large public contributions to the weather enterprise, the regulatory environment governing the functioning of each NMHS, set by national governments, differs significantly.

On the other hand, many developing countries struggle to provide sufficient public funds to enable a national capability in weather forecasting. This means that there are opportunities for private companies, universities and NMHSs to operate in a trans-national fashion. It is important in such an environment that the basic national infrastructure, as an essential contribution to the global observing system, is established and maintained in all countries.

In conclusion

The weather enterprise is today a global public-private partnership and indeed some estimates suggest that it is around a 50:50 partnership. Dependencies between the public and private sectors mean that they cannot survive acting on their own. Innovation in observing and computing technology and the existence of private capital is enabling the private sector component of the weather enterprise to grow rapidly. Indeed, the domain of global operational NWP is also seen as within scope of the private sector. But this can only be successfully built on the public sector investment in research and development in weather science and in the global observing system.

What is needed urgently is a constructive dialogue between leaders across the public and private sectors to co-design these developments for the benefit of the weather enterprise as a whole. It will be most beneficial for both sectors if collaboration rather than competition is the norm. Of course there are issues to be addressed and the WMO is the right organization to galvanize the sectors for mutual benefit (see text-box for the outcomes of the WMO Executive Council special dialogue⁶).

The urgency to do this comes from the need to devise more accurate and reliable weather forecasts to be even more effective in saving lives and protecting infrastructure at a time of increasing vulnerability to weather hazards due to changing climate. There is a need to convene a PPP Summit of leaders across the public and private sectors to address key issues of mutual interest.

Acknowledgments

The author would like to thank the WMO Secretary-General for inviting him to participate in the Executive Council special dialogue session on 15 June 2016. He would also like to thank Theresa Condor, Peter Platzer, Sylvie Castonguay and Richard Pettifer for helpful comments on a draft of this article.

Author

Alan Thorpe, Former Director General of the European Centre for Medium-Range Weather Forecasts (ECMWF)

References

Presentations to the WMO Executive Council special dialogue session on 15 June 2016

Bauer, P., Thorpe, A. J., and Brunet, G.: The quiet revolution of numerical weather prediction. Nature, 525, 47-55 (2015)

Bjerknes, V.: Das Problem der Wettervorhersage betrachtet vom Stadtpunkt der Mechanik und Physik, Meteorol. Z., 21, 1-7 (1904)

Pettifer, R.: The Development of the Commercial Weather Services Market in Europe 1970 – 2012, Meteorol. Appl. DOI: 10.1002/met.1470 (2014)

http://www.cubesat.org/about/

WMO Executive Council 68: EC-68/Doc.12(4) Public-Private Partnerships