Data Sharing for Sustainable Development: the WMO Information System (WIS) 2.0

Will the “cloud” and machine learning yield the next breakthrough in the weather, climate and water area? Exceptional advances in technology and its use over the last few decades have irrevocably changed how we work and live. Computers are more powerful than ever before; the smartphones in our pockets are faster than the mid-1980s Cray-2 supercomputer. Ninety percent of all data in existence has been created in the last two years. The cloud provides readily accessible capacity to store and process huge volumes of data. Machine learning is helping scientists to explore massive datasets and make new discoveries in fields from agriculture to genomics to particle physics. Fifty-five percent of the world’s population used the Internet in 2018 and there are more than three billion active social media users. What could that mean for the global weather enterprise?

Businesses increasingly harness data and technology to improve their performance. Often, these innovations also contribute to positive societal impact. For example, Global Fishing Watch promotes ocean sustainability by exposing illegal fishing. Machine learning algorithms are detecting patterns in vast sets of vessel-tracking data and satellite imagery to identify when a vessel is fishing. Global fishing activity is published in near real-time on the Web via a freely accessible interactive map. Now everyone can see who is fishing and where, unlocking unprecedented opportunities to improve fisheries management and support governments in safeguarding Marine Protected Areas. 

Meteorological data are vital factors to decisions about public safety, agriculture, health care, transport and more. By embracing these new technologies, National Meteorological and Hydrological Services (NMHS) can better control how their data are shared and with whom. They can leverage the data shared by the global meteorological community to deliver practical and effective services to their citizens and government.

Many NMHSs are already working with partners on modernization programmes that use new technology and more data to deliver improved services to end users. For example, the Mozambique National Institute of Meteorology (INAM) is developing an enhanced coastal modelling capability to provide improved early warning services to the fishing community and better mitigate the impacts of severe weather on trade and shipping through longer lead-times for preparedness. 

These kinds of data-intensive services benefit greatly from the increasing availability of new and more diverse data sources, easier access to computing resources to process large volumes of data in near-real time and use of Web technologies to convey the resulting insight directly to those who need to make decisions and take action. That said, without training and capacity development, the availability of new technology and data will not translate into better service delivery. 

The United Nations global agendas have reinforced the need for governments, private sector and academia to work together to respond to current and future societal challenges. Authoritative meteorological data are vital components in this endeavour. WMO Members need to increase the visibility and accessibility of data from their NMHSs while helping to raise their profiles as the authoritative voice on weather, water and climate. 

WMO is looking to the future through its draft Strategic Plan and Budget 2020-2023 and Governance Reform aimed at aligning form to function.  Both will be presented to the World Meteorological Congress in June 2019.Data and information systems play an important part in this, ensuring that WMO will be fit for a technologically advanced future and in enabling Members to effectively contribute to the mitigation of disaster risks and to sustainable development. 

Contributing to UN global agendas

• 2030 Agenda for Sustainable Development
• Paris Agreement on climate change

Our past

The WMO Global Telecommunications System (GTS) was established in the 1970s as one of the three pillars of the World Weather Watch (WWW). This world-wide coordinated telecommunication system was designed to enable Members to share data and products with each other in support of operational weather forecasting. Over the last 40 years, the GTS has successfully moved indispensable, time-critical data between NMHSs 24 hours a day, 365 days a year. Recognising the importance of meteorological data to everyone – not only NMHSs – the WMO Information System (WIS) was commissioned by Congress in 2007. WIS built on and incorporated the GTS, adding a data catalogue, data discovery portal and additional mechanisms for users to subscribe to and download data shared on the GTS. Members now operate centres that publish data and deliver services to meet national, regional or WMO programme requirements.

The issue

Despite enabling broader inclusion and use of data for all WMO programmes, and access to the data in external systems, WIS is perceived as a niche infrastructure that only offers data-sharing among peers in the expert meteorological community. Its focus is on the GTS and data supporting WWW, thus data and products from other WMO programmes and the broader meteorological community can be difficult to find. This means the full value of our data, and therefore our NMHSs, cannot be realised. 

Our future

 WIS 2.0 reaffirms the original intent of WIS: to open up access to environmental data and products. It will provide users with improved discovery of, and seamless access to, diverse authoritative information from a wide range of sources. These improvements contribute towards the WMO Reform aims of increasing the efficiency and effectiveness of the Organization

WIS 2.0 will increase the visibility and accessibility of NMHSs’ data, helping to raise their profile as the authoritative voice on weather, water and climate. Designed to support both free and open data as well as data released under commercial or restrictive licenses, WIS 2.0 will also make it simpler for NMHSs to keep control of their data. With better mechanisms to find and use meteorological data, WIS 2.0 aims to support NMHSs in delivering better services to their citizens and government. They will benefit from more data supplied by increasingly diverse and numerous sources.

The Governance Reform supports this more open way of working. The proposed Infrastructure Commission (Commission for Observation, Infrastructure and Information Systems) will cover all areas of activity and support Members by providing effective mechanisms for global data sharing.

Commission for Observation, Infrastructure and Information Systems 

The Commission shall contribute to: 

• Development and implementation of globally coordinated systems for acquiring, processing, transmitting and disseminating Earth system observations, and related standards; 
• Coordination of the production and use of standardized analysis and model forecast fields; 
• Development and implementation of sound data and information management practices for all WMO Programmes and their associated application and services areas.

Welcome to WIS 2.0

WIS 2.0 is an evolution rather than a revolution: the roles of WIS Centres and the majority of current WIS functions are largely unchanged. The essence of the change is to use the Web to share data and information. According to the World Wide Web Consortium (W3C) “the Web is the World’s most successful distributed information system.” The WMO review of emerging data issues finds that Web services are one of the technologies that “present new operating concepts that will improve operational efficiency, information sharing and service delivery, and enable users to more effectively exploit data.”

In WIS 2.0, participating centres will provide Web services that enable users to access and interact with data. Whereas the current WIS focused on operational data, the services which can be made available in WIS 2.0 also enable centres to share historical data and offer access to archives, where appropriate, to support the needs of all WMO programmes. Global Information System Centres (GISC) will continue to coordinate data sharing within their area of responsibility, and provide core services such as the discovery portal and catalogue management.

Real-time delivery of data and products in support of the WWW remains a core requirement for WIS 2.0. The existing data exchange mechanisms used on the GTS will be complemented with modern, Web-based messaging protocols such as those that underpin social media platforms like WhatsApp and Twitter.

The GTS already uses managed regional networks, known as ‘Area Meteorological Data Communications Networks’ (AMDCN), that use high-performance managed networks and Internet. With such networks, intermediate routing of messages between centres is no longer required: routing is delegated to the underlying network infrastructure to move messages from origin to destination. WIS 2.0 takes advantage of this change and will phase out the use of GTS routing tables and bulletin headers. 

With Web technology at its core, WIS 2.0 will use widely adopted open standards and leverage industry best practices. Firstly, this will allow the global user community to conveniently discover, access and use authoritative weather, water and climate information. Secondly, it will be simpler for NMHSs to partner with others to deliver services to end-users due to sharing the same base technology.

Providing data using the Web does not automatically mean that those resources are freely available to all without restrictions on use. Web technologies allow for authentication and authorization where necessary: the provider retains control of who can access published resources and can require users to accept a license specifying the terms and conditions under which those resources can be used before allowing them access.

The WIS 2.0 Implementation Approach proposes eleven technical principles to guide the evolution of WIS as outlined above.

Big data and cloud adoption

Satellites, radars and numerical models are producing more data than ever before, yet WMO Congress concluded in 2015 that most Members are “ill-prepared” for this change. Storage, management and processing of so-called “big-data” requires expensive infrastructure. Another challenge is that data volumes are becoming so large that it is impractical to download the data fast enough to meet operational requirements.

When designing their Web services, WIS centres publishing big-data should consider the capability of users to work with the data. Instead of expecting users to download entire datasets for local processing, WIS centres should provide Web services that allow users to download only the data or information they actually need. Such services may range in complexity – from extracting a geographic subset of data through to remote execution of a local area weather prediction model. In both examples, the data is processed on the data provider's infrastructure to create a product that is small enough to be conveniently downloaded and used.

Where data processing is complex, intensive or requires a lot of user-specific configuration, WIS centres should consider use of cloud technologies to underpin their services.

WIS 2.0 does not require the use of cloud; rather WIS 2.0 implementation will encourage WIS centres to adopt cloud technologies where it is appropriate to meet their users’ needs. There will not be technical regulation saying “cloud must be used.” Instead, we hope to see a gradual adoption of these technologies because it makes sense to do so. 

Cloud technologies are still new to many Members. Best practices on use of cloud and details of successful cloud implementations will be shared among Members. Additionally, the proposed WIS 2.0 implementation programme aims to provide capacity building opportunities to support cloud adoption.

Cloud implementation example: Copernicus Climate Data Store

On behalf of the European Commission, the European Centre for Medium-Range Weather Forecasts (ECWMF) have implemented the Climate Data Store (CDS) as part of the Copernicus Climate Change Service. The CDS is designed to provide Web-based access to, and interaction with, petabytes of existing climate datasets: observations plus global and regional re-analyses and climate projections. 

The CDS provides a “toolbox” that enables developers using a Web browser to write high-level python code to process CDS datasets and visualize the results, thereby creating specific applications that are delivered to end users as Web services. Crucially, this application code executes on a private cloud infrastructure with high-speed access to massive volumes of data, scaling elastically¹to meet users’ collective demands. Using the CDS, end-users can gain insight from climate data with minimal effort.^[1]

What happens next?

Feedback on the WIS 2.0 Implementation Approach has been gathered through consultation with Members. The implementation approach will be updated in response to that feedback and the new version submitted to Congress in June for consideration. 

Following Congress, further development of the technical requirements and architecture of WIS 2.0 is planned in collaboration with NMHSs that operate GISCs. This information will help Members assess the indicative implementation and operating costs.

Congress will be asked to authorize Executive Council to approve WIS 2.0 Implementation once this additional information has been provided for Members.