Which Hurricane Model Should You Trust?

Afternoon satellite images and data from the 2 pm EDT Thursday hurricane hunter mission into Invest 96L found that the storm had developed a closed surface circulation, but the storm had little in the way of organized heavy thunderstorm activity, and was not yet classifiable as a tropical depression. As we wait to see if 96L will develop or not, it is worth reviewing the performance of the models we use to forecast the track and intensity of tropical cyclones. Remember that all of these results are for storms that reached at least tropical depression status. For tropical disturbances that have not yet become a tropical depression, the intensity forecast models should not be trusted at all, and the track models will have much larger errors than analyzed here.

The National Hurricane Center (NHC) did a decent job forecasting Atlantic hurricane and tropical storm tracks during 2013, with errors similar to or slightly larger than the previous 5-year means for 12 to 96 hour forecasts, according to the 2013 National Hurricane Center Forecast Verification Report, issued in May 2014. A new record accuracy for 5-day forecasts was set, though forecasts for 12 to 96 hours were slightly worse than in 2012. One possible reason for the increase in track errors in 2013 was the small number of hurricanes we had (just two.) On average, the NHC track errors steadily decrease as the intensity of a tropical cyclone increases. The average track error in an official NHC 1-day forecast in 2013 was 57 miles, and was 118 miles for 2 days, 162 miles for 3 days, 190 miles for 4 days, and 190 miles for 5 days. The official track forecasts tended to have a westward to northwestward bias of 55 - 82 miles for 3 - 5 day forecasts (i.e., the official forecast tended to fall to the west or northwest of the verifying position.)


Figure 1. Verification of official NHC hurricane track forecasts for the Atlantic, 1990 - 2013. Over the past 24 years, 1 - 3 day track forecast errors have been reduced by about 60%. Track forecast error reductions of about 40% have occurred over the past thirteen years for 4- and 5-day forecasts. Image credit: 2013 National Hurricane Center Forecast Verification Report.

NHC Intensity Forecasts: a Notable Improvement in 2013
Official NHC intensity forecasts did better than usual in 2013, and had errors lower than the 5-year average error for all forecast periods (12 hours through 5 days.) Mean forecast errors in 2013 ranged from about 5 knots (6 mph) at 12 hours to about 13 knots (15 mph) at 5 days. Errors for 72 and 96 hour intensity forecasts were nearly 50% smaller than the 5-year average. The official forecasts were biased too high at most forecast times. There has been a noticeable improvement in intensity forecasts in the past few years, but this is likely due to the lack of rapidly intensifying hurricanes. These rapid intensifiers are typically the source of the largest forecast errors.


Figure 2. Verification of official NHC hurricane intensity forecasts for the Atlantic, 1990 - 2013. Intensity forecasts have shown little to no improvement since 1990. Image credit: 2013 National Hurricane Center Forecast Verification Report.

Which Track Model Should You Trust?
As usual, in 2013 the official NHC forecast for Atlantic storms was better than any individual computer models at most forecast time periods, although NOAA's HWRF model did slightly better than the NHC official forecast for 5-day forecasts. Once again, the European Center (ECMWF) and GFS models were the top performers, when summing up all track forecasts made for all Atlantic named storms. The two models were about equal in performance for 12-hour through 72-hour forecast, with the GFS model besting the European model for 3-day, 4-day, and 5-day forecasts. NOAA's specialized regional hurricane models, the GFDL and HWRF, were also respectable in accuracy for 12-hour through 48-hour forecast accuracy in 2013, though not quite as good as the GFS and European models. The simple BAMM model and CMC models did quite poorly compared to the others.

The best-performing model averaged over the past three years has been the European Center model, with the GFS model a close second. Wunderground provides a web page with computer model forecasts for many of the best-performing track models used to predict hurricane tracks. The European Center does not permit public display of tropical storm positions from their hurricane tracking module of their model, so we are unable to put ECMWF forecasts on this page. Here are some of the better models NHC regularly looks at:

ECMWF: The European Center's global forecast model
GFS: NOAA's global forecast model
UKMET: The United Kingdom Met Office's global forecast model (not evaluated by NHC in 2013)
GFDL: The Geophysical Fluid Dynamics Laboratory's hurricane model, initialized using GFS data
HWRF: The intended successor for the the GFDL hurricane model, also initialized using GFS data
NAVGEM: The Navy's global forecast model (which replaced the defunct NOGAPS model in 2013)
CMC: The Canadian GEM model
BAMM: The very old Beta and Advection Model (Medium layer), which is still useful at longer ranges

If one averages together the track forecasts from the first five of these models, the NHC official forecast will rarely depart much from it. These are the five models used to formulate the TVCA consensus model seen in Figure 3; the TVCA model was very close to the official NHC forecast.


Figure 3. Skill of computer model forecasts of Atlantic named storms in 2013, compared to a "no skill" model called "CLIPER5" that uses just climatology and persistence to make a hurricane track forecast (persistence means that a storm will tend to keep going in the direction it's current going.) OFCL=Official NHC forecast; GFS=Global Forecast System model; GFDL=Geophysical Fluid Dynamic Laboratory model; HWRF=Hurricane Weather Research Forecasting model; ECMWF=European Center for Medium Range Weather Forecasting model; TVCA=one of the consensus models that blends together up to four of the above models, plus the UKMET model (not shown); CMC=Canadian Meteorological Center (GEM) model; BAMM=Beta Advection Model (Medium depth.) The UKMET model was not evaluated. Data taken from the National Hurricane Center 2013 verification report.

Which Intensity Model Should You Trust?
Over the past three years, the official NHC intensity forecast has generally outperformed their four main intensity models. These four models were the dynamical HWRF and GFDL models, which subdivide the atmosphere into a 3-D grid around the storm and solve the atmospheric equations of fluid flow at each point on the grid, and the statistics-based LGEM and DSHP models (DSHP is the SHIPS model with inland decay of a storm factored in.) The top-performing global dynamical models for hurricane track, the GFS and European (ECMWF) models, are typically not considered by NHC forecasters when making intensity forecasts. The GFS model has done a decent job at making intensity forecasts over the past three years, but the European model has made poor intensity forecasts. In 2013 and for the period 2011 - 2013, the HWRF model was the best-performing intensity model for forecasts of 48 hours or less. The LGEM statistical model was the best one at longer-term intensity forecasts of 3 - 5 days.


Figure 4. Skill of computer model intensity forecasts of Atlantic named storms in 2011 - 2013, compared to a "no skill" model called "Decay-SHIFOR5" that uses just climatology and persistence to make a hurricane intensity forecast (persistence means that a storm will tend to maintain its current behavior.) NHC=Official NHC forecast; GFS=Global Forecast System model; GFDL=Geophysical Fluid Dynamic Laboratory model; HWRF=Hurricane Weather Research Forecasting model; ECMWF=European Center for Medium Range Weather Forecasting model; LGEM=Logistic Growth Equation Model; DSHP=Statistical Hurricane Intensity Prediction Scheme with inland Decay. Image credit: 2013 National Hurricane Center Forecast Verification Report.

Some Promising New Models From the Hurricane Forecast Improvement Project (HFIP)
Last year was the fifth year of a ten-year project, called the Hurricane Forecast Improvement Project (HFIP), aimed at reducing hurricane track and intensity errors by 50%. HFIP model performance in 2013 was improved over 2012. Notably, the 15-km resolution global FIM9 model was competitive with the top-tier GFS, European, GFDL, and HWRF dynamical models for track. The University of Wisconsin non-hydrostatic intensity model (UWN) performed well for intensity forecasts. Also, as detailed in a July 11 guest blog post by Dr. Morris Bender of NOAA's Geophysical Fluid Dynamics Laboratory (GFDL) in Princeton, New Jersey, and Dr. Vijay Tallapragada of NOAA's National Center of Environmental Prediction Environmental Modeling Center (NCEP/EMC), the intensity forecasts of the latest versions of the GFDL and HWRF hurricane models, when averaged together, make an intensity forecast that is one of the best available.

For those interested in learning more about the hurricane forecast models, NOAA has a 45-minute training video (updated for 2011.) Additional information about the guidance models used at the NHC can be found at NHC (updated in 2009) and the NOAA/HRD Hurricane FAQ (updated in 2013).

Sources of Model Data
You can view 7-day ECMWF and 16-day GFS forecasts on wunderground's wundermap with the model layer turned on.
Longer ten-day ECMWF forecasts are available from the ECMWF web site.
FSU's experimental hurricane forecast page (CMC, ECMWF, GFDL, GFS, HWRF, and NAVGEM models)
NOAA's HFIP model comparison page (GFS, ECMWF, FIM, FIM9, UKMET, and CMC models.)
Experimental HFIP models

This will be my last post until Monday. Hurricane expert Steve Gregory will be posting in my blog while I am gone the next three days.

Jeff Masters