Meteorological data, especially real-time weather forecast data, arouse considerable interest across
different sectors. But, how to explore a vast array of ECMWF open data?
Personal reflection
The ECMWF Open Data Explorer challenge aligned with my vision as I wished to create detailed
guidelines on how to use meteorological and climatological data to support various types of users from
operational meteorologists to data enthusiasts. Having the opportunity to translate ECMWF real-time
weather forecast data into information that drives action was one of the reasons I was so excited to
choose this challenge. The Code for Earth programme offered me an opportunity to realise my ideas in
the earth and computer science fields in order to contribute to open source innovations.
The successful outcomes of this challenge were a case study of a real-time forecast of the selected
parameter over the world over next seven days and one tutorial that can also be run on free online
platforms such as Google Colab and Kaggle. It was important to propose a technically feasible solution
and create a clear timeline, discuss it with the mentors, and follow it in order to achieve the ultimate
goal.
Project journey
There was no all-in-one solution where one could find all the requested information to process ECMWF
real-time weather forecast data. The main objectives of this project were to improve findability,
accessibility, interoperability, and reusability (FAIR) of ECMWF open data.
The key deliverable of this project was the one-stop shop Open Data Visio. The tutorials provide the
scientific and user communities with the state-of-the-art tools and assist them in extracting tailored
information from a wide variety of forecast data and integrating it into their workflows.
The ECMWF Open Data Explorer is supported by Jupyter notebooks that are structured in the following
order:
- Setting up the environment,
- defining parameters to retrieve from open datasets,
- using the ECMWF open data API,
- manipulating the data, and
- plotting the data.
Project spotlight
ECMWF open data include meteorological information from the Integrated Forecasting System (IFS) and
Artificial Intelligence Forecasting System (AIFS) models. However, not all AIFS and AIFS ENS (the
ensemble version) parameters are open to the public. ECMWF real-time open weather data are available from different locations, including ECMWF Data Store, Amazon’s AWS S3 Buckets, Microsoft
Planetary Computer, and Open-Meteo. When retrieving data from the Open-Meteo data provider, a
user has to specify the exact coordinates and time interval.
The ecmwf-opendata package simplifies the download of ECMWF real-time weather forecast data from
Amazon AWS and ECMWF Data Store. It also enables data retrieval from multiple models, parameters,
and dates. Some obstacles a user might encounter are changes in the file-naming convention when
accessing historical data. Since Microsoft Azure has changed the anonymous access function of the
ECMWF open data, it is not possible to download data from Microsoft Azure Storage with this package.
The actual data assets are stored in private Azure Blob Storage containers. Alternatively, one can access
data using the pystac_client library.
The Earthkit package is a powerful tool for speeding up weather and climate science workflows.
Nevertheless, a user might experience difficulties when using the stream and parts option. The parts
option can be specified in order to retrieve data only for a specific parameter from the selected data file.
earthkit-plots is the visualisation component of earthkit and also enables automatic data styling.
However, it is not possible to define a custom plotting style to all earthkit plots. The earthkit package is
in development and subject to ECMWF guidelines on Software Maturity. Therefore, additional features
will be added in the future.
For more information, visit https://ecmwfcode4earth.github.io/open-data-explorer/.
Behind the scenes
As a one-woman show, it was both inspiring and challenging to work on this project. I was not faced
with collaboration challenges that some teams might have had. On the other hand, I lacked
brainstorming sessions with a team.
For this project, I used the following tools: Jupyter to create a Jupyter Book and Python with earthkit,
ecmwf-opendata, pystac-client, planetary-computer, and openmeteo_requests packages. During the
Coding Phase, I received some practical advice from Milana Vuckovic, Christopher Polster, Maartje
Kuilman, and Yigit Altintas, and relished the challenge of exploring ECMWF open data and documenting
best practices for data analysis. I would definitely recommend the Code for Earth programme to early
career and experienced professionals with an interest in weather, climate, and/or data science.
Conclusion
The experienced mentors from ECMWF stimulated me to create a meaningful environmental story,
along with an open source solution using innovative approaches.
ECMWF seasonal forecasts are open to all since the 1 st of October 2025 and thus another project
opportunity arises there. Additional case studies can offer fresh insights into the development of the
earthkit package and support data analysis projects among scientific, commercial, and operational users.