Artificial intelligence: How AI programmes influence the weather forecast

Artificial intelligence (AI) is currently on everyone's lips, whether it's images, videos, music or other things. In everything, attempts are being made to train computers to achieve better results than those achieved so far and to make our lives easier overall or at least make work processes more efficient.

The most prominent example is Chat GPT. Now in its fourth version, this software tool not only spits out texts, but can also hold real conversations. AI is also used in meteorology. My colleagues from the IT department already like to use it regularly to analyse, generate and improve programming codes. And it works wonderfully!

Computers are therefore trained using existing knowledge. If there is a specific enquiry that it may already know, the machine estimates which answer is most likely. The results often fit quite well or get better and better from query to query.

Also interesting:

• Artificial intelligence: New tool enables prediction of deadly monster waves
• Apps for sailing: The top 10 weather and routing programmes
• Meteorology: Which parameters influence the weather for sailors

When it comes to estimating probabilities, we almost inevitably end up in weather forecasting. For a long time, various models and many individual runs have been used to estimate what the weather will be like in the coming days. A very chaotic system that is sometimes able to spit out accurate forecasts over ten days, but sometimes for no longer than 24 hours.

Meteorology offers itself as a test field for AI

Weather patterns basically repeat themselves and primarily follow the atmospheric rules of thermodynamics. So the basic framework is in place. This is what Google and the European Centre for Medium-Range Weather Forecasts (ECMWF), among others, thought.

Google has the infrastructure required for AI calculations in the form of mainframe computers, while the ECMWF has vast amounts of meteorological data. In combination, the GraphCast weather model was created. This model, which is still experimental, was fed with so-called re-analysis data. In other words, measured values were mixed with model data in order to arrive at the global weather patterns of the past decades. Until around 30 years ago, the measurement network was not as dense as it is today.

In this way, the weather situation was recalculated backwards. The GraphCast model was then fed with the global weather from 1979 to 2017. It learnt how meteorological patterns behave and what weather results from this for the following days in various places around the world.

To start a weather forecast calculation, all you really need are the patterns of the global pressure distributions of the previous six hours and the current situation. Together with what has been learnt, this is enough to obtain a forecast for the following hours.

However, a press release from Google then electrified the experts. Firstly, because the AI-generated model has a hit rate of over 90 per cent. We meteorologists were even more amazed, as it takes less than a minute of computing time to produce such a reliable forecast for the next ten days! By comparison, the previous operational weather models take around four to six hours to produce the result.

Superiority in forecasting extreme weather conditions

How does that work? Well, in the ECMWF model, 52 individual runs are started simultaneously, each run with small changes in the initial conditions. An estimate is then made after up to six hours. The lines that are closest also show the most likely path of the weather systems in the coming days. What you can see on Windy, for example, as an ECMWF model, is therefore nothing other than the most probable result of 52 individual model calculations.

You can also display all 52 individual forecasts, especially for local forecasts. This can result in a temperature of 30 degrees for one race in Hamburg, for example, while another forecasts only 15 degrees for the same day. All other races lie in between, so there is a high probability that it will be around 20 degrees.

It is now even possible to view the results of individual runs not only for a specific location, but also for large regions. This means that with just one single weather model, you would have 52 different weather maps available - from which you only have to choose the most probable one.

Admittedly, this all sounds a little complicated, which is why the long computing time is needed. GraphCast was therefore tested against this ECMWF model - and thus against the most reliable one to date. In terms of accuracy for the parameters wind, temperature and air pressure, it was immediately superior. But apparently it was also better at forecasting extreme weather conditions, even though it had not yet been trained for this!

However, two relatively coarse weather models were compared with each other. GraphCast was operated with a grid size of 28 by 28 kilometres and the ECMWF model was also used with exactly this grid size. This means that there are only points that spit out results every 28 kilometres in longitude and every 28 kilometres in latitude. This is good for predicting the paths and development of highs and lows, but not, for example, local phenomena such as sea winds.

Models for long-term forecasts will be more stable and reliable

Nowadays, however, it is possible to achieve a much smaller scale. The ICOND2 model of the German Weather Service has a resolution of 2.1 kilometres. So far, this is still far ahead of AI. However, if GraphCast or other AI models also advance into these resolution spheres or become even smaller-scale, it will certainly be possible to predict more regional events such as thunderstorms and perhaps tornadoes better than at present.

As if that wasn't enough, Google has also co-developed extensions to GraphCast, such as NeuralGCM. This is a model that combines AI and the traditional physics-based method of weather forecasting: Large-scale atmospheric phenomena are simulated and, at the same time, smaller features such as clouds and precipitation are accurately estimated using neural networks. This should lead to more stable and reliable models for long-term forecasts.

The software giant has also linked another tool to the AI: Scalable Ensemble Envelope Diffusion Sampler (SEEDS) is the name of the method, which, as mentioned above, also allows many different forecast runs to be calculated in order to estimate which scenario is most likely to occur. This is primarily used to recognise extreme weather events at an earlier stage.

Although this is not a new method in principle, these many prediction variants are also generated in just a fraction of the previous computing time, mainly due to the AI. Anyone who would like to get an impression of the accuracy already achieved by the GraphCast model can easily do so. The project's daily weather maps are available on the ECMWF website and can be compared with those of the operational model.

See what should happen every 15 minutes thanks to AI

If AI applications make further inroads into the weather scene and replace the existing numerical models - and they will! -this would have enormous consequences. In particular, the computing capacity saved would be available for other applications. For example, the French weather service has launched a new model, Arome PI, which not only calculates the current weather situation every hour, but also every 15 minutes. This means that it should become more and more precise and refined in order to see where rain showers are heading, for example, and whether and, above all, where new ones are forming.

So far, the development of shower cells and thus possibly also of heavy rain with flooding has been like a lottery: I always compare the forecast of where the next shower will occur with boiling water in a pot. Here too, nobody can say where the next bubbles will rise.

So you can already click through the weather maps and see what is supposed to happen every 15 minutes - in the hope that this will bring you even closer to reality. This will be an extremely important tool, especially for severe weather warnings.

China is not idle when it comes to AI and weather either. The company Huawei Cloud is trying to train its own model called Pangu-Weather for seven-day forecasts. The university in Beijing, on the other hand, is focussing on small-scale precipitation forecasts for just three hours in advance using the AI model FourCastNet. Here, too, the models benefit from the statistics of recent years.

But this also has its pitfalls. Especially in extreme weather situations, which we are seeing more and more frequently around the world. Such events are not yet included in the statistics. Hurricanes and typhoons with categories 3 to 5, known as major hurricanes or monster typhoons, are also still too rarely represented in the data for them to be reliably predicted. In addition, the physical equations used by the models used to date are also mutually dependent. Certain processes such as clouds, radiation and turbulence are therefore interdependent and difficult to model. Taking this into account is not easy, especially with a model resolution of 28 by 28 kilometres.

Meteorologists will still be needed for evaluation in the future

The approaches are therefore very different. Pangu-Weather only used observational data as a basis. FourCastNet, on the other hand, which works on a much smaller scale, uses both AI and physical equations and combines the two. Pangu-Weather is said to calculate 10,000 times faster than comparable numerical models. It is therefore suitable for medium-term forecasting, but does not make any statements about precipitation. This is where the other model comes into play, which, the closer you get to the weather event, allows a more accurate forecast of the amount of rainfall.

The fact that we are already talking about two models that are used for different applications indicates that meteorologists will still be essential for a proper assessment of the situation in the future. AI is not new per se, and sailors have already been using it to a certain extent for several years. Especially when it comes to routing in combination with weather forecasts.

This involves working with software tools such as "Expedition" or "Adrena" based on the polar diagrams of your own boat. At the end, the programmes spit out what is probably the best or fastest route, taking wind and weather into account.

They move the virtual ship forward hour by hour and look at the prevailing wind at the location they have reached at that time. The polar diagram shows the wind direction and strength at which the boat is sailing fastest. So-called isochrones are output time step by time step - with the corresponding suggested course.

Professional sailors do nothing else several times a day when new weather data comes in. Current data can also be included. This allows the software to estimate the best possible course. However, these calculations take time. Especially if you enter a lot of data about your ship.

The whole thing has also been used in commercial shipping for some time. The AI is supposed to indicate the optimal and safest route for freighters or tankers as quickly as possible and without the need for human weather routers. This saves fuel and therefore costs, not least with the help of wind, waves and currents. At the same time, you avoid getting caught in storms and possibly losing valuable container freight.

Not least thanks to Starlink Internet on board, these calculations are no longer carried out on the on-board computer, but are outsourced to external servers. The current and historical weather data is already stored there, which means that the individual calculations for a specific ship can be carried out even more quickly.

This is what the future already looks like. But we can't do without people and their expertise. Despite all the technical possibilities that already exist, my colleagues and I are still the number one point of contact for captains when they want to correctly assess weather conditions and work out the best route to the next port of destination.

And what does AI do for us sailors?

It is highly likely that more and more apps will be able to create automated routing in the future. The user then simply enters the start and destination, for example from Kiel to Rostock, and adds the desired start time. The app then calculates the best course for the most likely weather conditions.

What is the consequence? Will autonomous sailing, controlled and monitored by AI, inevitably come soon? It will be interesting to see. However, relying completely on technology harbours risks. Particularly because the weather system is so complex and chaotic that it will never be possible for everything in it to be accurately modelled. Uncertainties remain. Especially when it comes to difficult weather conditions where several models have to be consulted for a reliable forecast.

Once again, extreme weather conditions should be mentioned here. It will certainly not be possible to categorise them precisely for some time to come without the knowledge and many years of experience of meteorological experts.

And with the routing tools in particular, which many people already like to rely on, most people don't realise that often only the mean wind is taken into account. Gusts or other severe weather parameters are not taken into account. If you then download the weather data, which only has a temporal resolution of three hours, for example, you may not even see an approaching severe weather situation.

Knowledge of basic weather formation and development is therefore still essential - as is working with it properly. But you should also know how to use all these new, fascinating tools. After all, the rapid pace of technological development cannot be ignored. With the intention of improving weather forecasts, it will change the world for the better, at least in this area.

I myself am excited to see where the journey will take us in the coming years - and I definitely want to be part of it.

Conventional and AI-generated predictions in comparison

Under charts.ecmwf.int all data products of the European Weather Centre ECMWF can be viewed on the Internet. There you will not only find the model forecasts of the conventional operational and numerical model, which is also used, for example, in Windy.com or other private and commercial weather apps. Instead, at the bottom of the very full page, you can also find the new AI models and the results they produce: While there are hardly any significant deviations in a relatively short-term forecast in the region of around 24 hours, the differences are significant in a longer forecast period.

Above is an example of the weather maps based on a ten-day forecast for the pressure distribution over Europe from 31 July to 10 August 2024: While the numerical model already sees a very strong low off the coast of Ireland, the two AI models also forecast the formation of this low, but with a significant delay and, above all, much further west, with the core between Newfoundland and Greenland.

How reliable are the old and new weather forecast models?

The new AI models are constantly compared with the previous ones. Their behaviour too. For example, model activity is monitored. If the graphs run evenly, then few surprises can be expected in the forecast over a period of ten days. Only the FuXI (another AI model) in the example shows very chaotic behaviour from day five onwards. In contrast, all other models, both the established and the new AI models, show a relaxed drop in the reliability of the forecasts. Incidentally, the 500 hPa values, which are measured at an altitude of around 5.5 kilometres, are particularly important here. They provide the best indication of how highs and lows will behave down to the ground over the next few days.

The fact that most AI models, as in this example, show similar behaviour to the established AIFS and IFS models of the ECMWF, and that some of them are already more accurate and less error-prone, is simply astonishing.