4 min readForecasting

Forecast verification: how to score a weather forecast

A forecast you have not scored is a forecast you cannot price, tune or trust. The common scores in plain language, the pitfalls, and one published example of honest reporting.

Forecast map of wind speed 80 meters above ground over Washington and Oregon on a 1 km grid, with bright streaks of strong wind through mountain gaps.

First published November 11, 2026 · Updated November 16, 2026

Forecast verification is the practice of scoring forecasts against what actually happened. It is how a forecast-team lead learns whether a new model setup is an improvement, how a buyer learns what a vendor's forecast is worth, and how an operations lead decides how far to trust tomorrow's number. This guide covers what to compare against, the common scores in plain language, the pitfalls that produce flattering results, and one published example of a result reported with its limits attached.

What to compare a forecast against

No reference is perfect truth. Each has strengths and blind spots.

  • Surface stations measure temperature, wind, pressure and rain directly at a point. They are dense near airports and cities and sparse elsewhere, and standard wind measurements are taken at 10 m, far below a turbine hub.
  • Lidar measures wind at several heights above the instrument, including hub height. It suits wind energy well, but few sites have one.
  • Radar sees rain, hail and storm structure over a wide area every few minutes. It measures energy reflected by precipitation in the air, not rain reaching the ground, so turning it into rainfall takes care.
  • Analyses are gridded best estimates of the atmosphere that blend observations with a model. They cover everywhere, which is convenient, but they carry the habits of the model that made them. A model scored against its own analysis is flattered.

Whatever the reference, score a baseline too: persistence (no change from now), climatology, or a public forecast such as HRRR, NOAA's hourly US forecast model. A forecast earns its cost only by beating what you could have had without it.

The common scores in plain language

Take five wind forecasts that missed by +2, −1, +3, −2 and +3 m/s (forecast minus observed). The numbers are made up to show the arithmetic.

  • Bias is the average error with signs kept: (2 − 1 + 3 − 2 + 3) ÷ 5 = +1.0 m/s. The forecast runs high. Bias is the easiest error to correct, but a bias of zero can hide large errors that cancel.
  • Mean absolute error (MAE) is the average size of the miss with signs dropped: (2 + 1 + 3 + 2 + 3) ÷ 5 = 2.2 m/s. Read it as the typical miss.
  • Root mean square error (RMSE) squares each error, averages the squares and takes the square root: √((4 + 1 + 9 + 4 + 9) ÷ 5) = √5.4 = 2.32 m/s. Squaring makes large misses count for more, so RMSE is never smaller than MAE. Use it when one large miss costs more than several small ones.

For events, such as a storm at a site or wind above a turbine's cut-out speed, count hits, misses and false alarms. Say a storm was forecast on 20 days and arrived on 12 of them, and 3 more storms arrived without a forecast (made-up numbers again).

  • Hit rate, also called probability of detection: hits ÷ (hits + misses) = 12 ÷ (12 + 3) = 0.80. Of the storms that happened, 80% were forecast.
  • False alarm ratio: false alarms ÷ (hits + false alarms) = 8 ÷ (12 + 8) = 0.40. Of the storms forecast, 40% did not happen.

Report the two together. A forecast that calls a storm every day has a perfect hit rate and a useless false alarm ratio. Fuller definitions are in the methods guide kept by the Joint Working Group on Forecast Verification Research, a working group under the WMO's World Weather Research Programme. METplus, from the Developmental Testbed Center with NCAR and NOAA, is a public verification framework.

Three pitfalls that mislead

Representativeness

A model value is an average over a grid square. An instrument measures one point. A single grid square can hold both a ridge and a valley. Height matters as well: an anemometer at 10 m says little about wind at 80 m. Part of the "error" is this mismatch, not the forecast.

Timing errors and the double penalty

A high-resolution model draws a sharp storm. If it puts that storm 20 km away or an hour late, a point score counts a miss where the storm was and a false alarm where it was forecast: two penalties for one near miss. A smooth, coarse forecast that never commits can score better while being less useful. Scores that allow a neighborhood in space or a window in time treat sharp forecasts more fairly.

Cherry-picked cases

One dramatic case proves nothing. Fix the period, the sites and the score before looking at results. Then report every case, the sample size, the baseline and what did not work.

A worked example of honest reporting

Aerava, the weather intelligence platform operated by TempoQuest, publishes a wind comparison. Its parts line up with the checklist above.

  • Sample: twelve nights at one Oklahoma wind farm.
  • Reference: public DOE lidar at 80 m.
  • Baseline: HRRR alone.
  • Result: the 1 km forecast blended with HRRR had 11 to 13 percent less error than HRRR alone, and was better on 11 of 12 nights.
  • What did not work: on its own, the 1 km run did not reliably beat HRRR.
  • Status: the blend is research and is not on sale.

Aerava also says what it has not scored: storm forecasts are not yet scored, and hail size is experimental guidance. Twelve nights at one site is a small sample, and the statement claims no more than those twelve nights. That is the pattern to ask of any vendor, TempoQuest included: the sample, the reference, the baseline, what failed and what is still unscored.

TempoQuest is developing the Forecast Verification Tool (FVT), designed to improve the verification of weather forecasts. The scores and pitfalls in this guide are general practice, not a description of FVT.

What to do next

Read what Aerava's 1 km site forecasts are, or contact TempoQuest.

Your forecast

Have a forecast question of your own?

Tell us the decision it has to serve. One short form reaches the team, and we review every request.