WMO Data Initiatives and the Broader [UN] Data Agenda

WMO has been a leader in the facilitation of global data exchange and exploitation since its inception. Data exchange is at the heart of the WMO Convention, defining the core mission and purpose of the organization. The free and open exchange of meteorological observations dates back to 1873 with the creation the International Meteorological Organization, the predecessor to WMO (see article 1). But in today’s fast-changing world, WMO’s head start on data exchange is at risk of being lost and a shift in its approach is urgently needed to ensure its Members are not left behind.

In the last few decades, the rest of the world has realized the value of data, creating a groundswell of enthusiasm across all sectors of endeavor for the gathering, analysis and, often, monetization of data. WMO meanwhile has continued to work with its Members to facilitate and grow the amount of data available to the international community – but not, it must be said, at the same pace as the rest of the world.

Data issues have also arisen among Members. Some NMHSs – facing financial difficulties or recognizing the value placed on meteorological and climate data – have joined the move to monetize their data and outputs, selling derived products and in some cases the observations themselves. This can be a source of tensions among Members in an Organization that is built on the free and unrestricted exchange of data. The authors would argue that the real value of Members data is the downstream economic value that is created through analysis and prediction products, and impact-based services to assist decision-makers. Making data available, from the perspective of a government, increases the downstream societal and economic value generated. A good example is the opening of the Landsat archive by the US Government which in 2011 generated an estimated US $1.7 billion in economic benefit for US users alone [1].

Today, the global focus on data provides a number of opportunities to WMO and its Members. The data landscape has opened up through rapid technological development and the societal and economic benefits of open data arrangements have been recognized. This article explores this change in terms of the WMO relationship with global technology players and the potential implications for WMO Members. It also looks at the new WMO Unified Data Policy in the context of the global data agenda and the UN Secretary-General’s Data Strategy.

WMO Data Policy and UN Secretary-General’s Data Strategy

As a specialized agency, WMO is part of the United Nations (UN) family, which also deals with the global data agenda. The UN Secretary-General is leading the way at the highest level in setting a strategy framework for better use of data, following approaches that are fully grounded in UN values such as human rights. The UN Secretary-General’s Data Strategy 2020–2022 is a “strategy for data action by everyone, everywhere in the UN family – for insight, impact and integrity.” It is the UN’s agenda for a data-driven transformation, focused on building the data, digital, technology and innovation capabilities that the UN needs to succeed in the twenty-first century. Data permeates all aspects of work undertaken within the UN system. The power of data, harnessed responsibly, is critical to the global agendas it serves. The UN Data Strategy seeks to capitalize on the UN family’s footprint, expertise and connectedness to create unique opportunities to advance global “data action” with insight, impact and integrity.

Starting with a vision of the data-driven organization, the strategy is built on three core pillars:

1. Setting strategic foundations – In building a whole-of-UN data ecosystem that maximizes the value of data, the UN is striving to unlock its full potential. It seeks to make better decisions and deliver stronger support to people and the planet – in the moments that matter most.
2. Create value with data and focus on priorities – This pillar is based on delivering use cases that demonstrate the value added for stakeholders in accordance with UN priorities, which include the Sustainable Development Goals (SDGs), Climate Action and Gender Equality – all of relevance to WMO.

Figure 2 - UN Secretary-General's Data Strategy Model

1. Foster enablers, nurture capabilities and iterate – This involves adopting a learning-by-doing approach to foster stronger enablers (people and culture, data governance and strategy oversight, partnerships, technology environments) and to build new capabilities in an iterative and agile fashion. There will be special focus on analytics (what happened, why it happened, what may happen next, and how to respond) and data management (to ensure everyone can discover, access, integrate and share the data needed to fulfill our responsibilities to the organization, people and planet).

The strategy recognizes the need to build partnerships which connect better with global data ecosystems. There is a strong synergy with the WMO Data Policy of free and unrestricted exchange.

The UN’s data strategy acknowledges that the organization is at the start of a long journey, and that it will take some time before its data-related capabilities are truly transformed across the UN family. Having already established a foundation of open data sharing, the WMO is on the same journey but at a different point. Nevertheless, it faces some of the same challenges. For WMO, greater collaboration and partnership on data offer opportunities for the Organization and its Members to play a larger role in addressing the global challenges and to provide better impact-based services at home.

The WMO Strategic Plan 2020-2023 focuses on an integrated and comprehensive Earth systems approach, with data at its heart. This core goal is supported by the advanced and advancing capabilities of WMO Integrated Global Observing System (WIGOS), WMO Information System (WIS) 2.0 and the seamless Global Data-processing and Forecasting System (GDPFS). WMO has also come to appreciate the need to broaden its collaboration and partnership opportunities.

Within the UN system, WMO has strong collaborative relationships with many organizations, specialized agencies and programmes. The International Oceanographic Commission (IOC) of UNESCO is a key collaborator and partner that is aligning its own data policies with those of WMO. WMO is also building a strong collaboration on research and services with the World Health Organization (WHO), with potential under the WMO Unified Data Policy to further develop data exchange between the two organizations as new service requirements are identified and prioritized.

WMO is working with its partners to integrate meteorological, climatological, hydrological and environmental data with demographic, health and other data from UN partner agencies. This will provide improved guidance to decision-makers and address the strategic priorities of the broader data agenda within the UN.

The move to broaden, reframe and reassert its data policy via the Unified Data Policy is motivated to deliver what WMO needs to serve global challenges and the mandates of its Members. These goals align with the ambitions of the UN Secretary-General’s data strategy.

The Unified Data Policy reflects that international data exchange within WMO and with its partners supports multiple global agendas. Weather, climate, water and related environmental data and services are essential to the implementation of the SDGs, the Paris Agreement under the United Nations Framework Convention on Climate Change (UNFCCC) and the Sendai Declaration and Framework for Disaster Risk Reduction 2015–2030.

WMO Data Policy and the global data agenda

It is often stated that we live in a data-driven world. Algorithms built on data feed our news cycle, suggest movies for us to watch, curate our social media and select the advertisements we see. Whilst WMO has historically been a leader on the free and open exchange of data, it is fair to say that the global discussion on data has progressed quickly, creating new technologies, terminologies and tools in the process, overtaking WMO. Whilst WMO continues to work with its Members to facilitate and grow the amount of data available to the international community, it is no longer in advance or moving at the same pace as the rest of the world.

On the other hand, the machinery of WMO needs to be stable and reliable for three reasons. Firstly, it is essential to closely manage and track the impacts of any changes in observing technology and practices to monitor long-term trends in the climate. To that end, there is a tendency to conservatism that has influenced many aspects of data acquisition, management and sharing. Secondly, WMO technical regulations and resolutions are made and implemented by consensus and WMO can be limited to some extent by the capabilities and capacity for change of its Members. Thirdly, to enable prompt and effective life-saving decisions, such as issuing targeted warnings to prepare communities when severe weather events threaten them, it is critical to have trusted data available to underpin models and services.

Figure 3 - Weather station Krosno, Professor and graduate student preparing a weather balloon, Taranis – Bull Sequana XH2000, Meteo France (Ranked 49 in the Supercomputer Top 500, June, 2021)

The global data landscape and the technology giants

Data policy and the exchange of data between Members is enabled by technology, and the rise of data as a commodity has driven, in a synergistic way, enormous advances, from cloud computing to artificial intelligence. Data handling systems are commoditized to the extent that these capabilities are available to almost anyone with an Internet connection. And with this technological innovation has come a level of disruption to traditional markets, the emergence of new commercial opportunities and many new enterprises, both big and small. Amongst these, there are clear market leaders who dominate the industry and provide the bulk of the "as a service” segment.

Amazon is the market leader for cloud-based solutions, followed by Microsoft and then Alibaba and Google (Alphabet) (Table 1). Apple and Facebook dominate their own segments, but are not currently significant players in cloud-based computing services. IBM also holds a strong market position but more as a supplier of hardware and services such as those delivered by the Weather Company, its subsidiary.

Table 1 - Worldwide laaS Public Cloud Services Market Share, 2019–2020 (millions of US$) (Source: Gartner (June 2021))

Company	2020 Revenue	2020 % Market share	2019 Revenue	2019 % Market share	2019–2020 % Growth
Amazon	26 201	40.8	20 365	44.6	28.7
Microsoft	12 658	19.7	7 950	17.4	59.2
Alibaba	6 117	9.5	4 004	8.8	52.8
Google	3 932	6.1	2 367	5.2	66.1
Huawei	2 672	4.2	882	1.9	202.8
Others	12 706	19.8	10 115	22.1	25.6
Total	64 286	100.0	45 684	100.0	40.7

Social media and Internet search engines play an important role in the provision and dissemination of weather information. The latest data shows that there were 4.48 billion social media users around the world in July 2021, some 57% of the total population. Nearly 93% of all web traffic transits through search engines and more than four in five Internet users access news through social media.

This trend is only going up. In 2020, Facebook provided a regular source of news for about a third of Americans. According to one study, nearly 65% of Google searches ended without a click to another website – up from 50% in June 2019. Google’s vision statement is “to provide access to the world’s information in one click,” and the results are frequently provided as a predictive search, that is, giving information before users launch a search query. For example, if one clicks “weather” after typing “w,” Google provides a large amount of weather data for the user’s location at the top of the results: current conditions (temperature, precipitation and wind). Many users undoubtedly find what they need without having to click through to where the data is sourced – weather.com or the website of the local NMHS which may be located further down the list of results.

According to a report from the International Data Corporation (IDC)[2], there is an important strategic shift among leading public cloud Infrastructure as a Service (IaaS) providers towards development of a range of dedicated and hybrid cloud deployment options to meet enterprise demand for high performance and distributed edge use cases. The public cloud IaaS market experienced tremendous growth during 2020, expanding 34% to US$ 65.5 billion.

With such large players dominating the market and providing economies of scale, it is not economic for NMHSs to try to “go it alone” on cloud computing, for example. Whilst there are plans for a “European Weather Cloud”[3] supported by ECMWF, EUMETSAT and the NMHSs of their Member States, this is out of reach for the majority of WMO Members. Similarly, for distribution of information, it is impossible to compete with the big players in terms of reach and market power.

The great data challenge

The global weather machine[4] produces enormous volumes of data each day. With each increase in computing power comes heightened spatial and vertical model resolution, driving improved performance and massive increases in data volume. For example, ECMWF produces 120 Terabytes (TBs) of raw weather data daily and 30 TBs of user-defined products, the equivalent of a 144 TB portable hard disk or a Petabyte (PB) per week[5].

In terms of delivery to users, the average data transmission volume handled by the ECMWF Production Data Store (ECPDS) is approaching one PB per month. ECMWF forecast products are disseminated to 547 places in 78 countries, and observational data are retrieved from 557 places in 34 countries.

The large volume of data creates a problem. It is not practical to shift entire datasets around using Internet connections. For many WMO Members, the problem is exacerbated by limitations in their own storage and processing systems, and in many cases slow and possibly unreliable Internet connections. The increased capacity of computing systems and the desire of many NMHSs for higher resolution products will further aggravate this problem.

The global nature of both weather and the technology sector brings another issue. Developers of technology – such as mobile telephony and smart watches – may wish to incorporate weather information into their products. A simple example would be an “active” smart watch that tracks exercise, provides weather information to its user and integrates weather information to compute the user’s level of exertion or other derived data. In such a case, a separate weather interface would not be developed for each country or territory; instead the developer would look for a globally consistent source of data that they can be accessed in a standard way. The simplest delivery method would be an Application Program Interface (API) that delivers the data required to meet the needs of the user through a simple query. For example, the API would provide current conditions or a weather forecast for a specific location.

There is no existing global platform providing real-time and forecast information from individual NMHSs. It would also be impractical for developers to cobble together inputs from the individual websites of NMHSs that have no consistency in format or delivery. The simple solution for developers is to go to a single global source of information such as The Weather Company. Developers seeking more detailed information may source data from, for example, National Oceanic and Atmospheric Administration’s (NOAA) freely available products. However, this may come at the expense of ease of use as files are in scientific formats and some are very large.

This creates a problem for NMHSs. Though they often hold the most locally accurate products and data relating to their territory, they are simply too small for the app developers, who may also be too small to deal with NMHSs individually, particularly when markets are global. How then can the individual NMHS get its information to users without creating its own app?

 

Bringing it together

The evolution of the global data phenomenon and the rise of the tech giants creates opportunities for WMO and its Members. Too big to ignore and with technical capabilities that can benefit WMO Members, the large companies can be part of the solution.

What has become clear is that WMO and its Members can be direct beneficiaries of the opportunities available. One example would be in recognizing the important role of the tech giants in providing the underpinning technology for data exchange and for enabling the efficient distribution of information and services for sound decision-making.

The opportunities are perhaps even more compelling at the global level in the pursuit of efficiency and effectiveness, while building on the stability, reliability and trust that are so important to meeting Members’ service commitments at a national level. For example, NMHSs could be consumers of cloud services for computing and storage; but there is also a further opportunity here to create collaborative spaces where NMHSs can work together on large datasets. What is clear is that strong engagement with the sector is required and, whilst there will be a need for engagement by individual NMHSs, this will be enhanced and the full benefits realized only through coordination at the global level.

Similarly, the growing role of data in international policy means the time is ripe for WMO to benefit from greater collaboration and partnership within the UN. The broader data strategy within the UN is aligned with, and supports, the free and unrestricted exchange of data under the WMO Data Policy. The increased availability of ancillary data – such as impacts of disasters, health information and other data held by UN agencies – will enhance the ability of WMO Members to provide impact-based services by combining meteorological, climatological, water and environmental data with these datasets. At the same time, WMO has an important role to play in providing data to support larger UN agendas – from disaster risk reduction and climate change to sustainable ocean management and global public health.

Footnotes