Model Muttering
Note: This post is rated "G", suitable for all audiences; does not contain the weather "s" word.
Weather model skill as a function of year, from The Emergence of Numerical Weather Prediction, Peter Lynch, Cambridge University Press, 2006. Chart © ECMWF. Click image to enlarge.
In the nearly 60 years since the modern era of numerical weather prediction began, many orders of magnitude improvements have been made in both computer technology and the formulation and methods of solution of the equations describing the atmosphere. As a result, huge improvements have been made in the accuracy of daily weather forecasts. As good as the models have become, however, it's important to realize what they can and cannot do.
First of all, not all variables are created equal. Of the various parameters of interest, some of them can be predicted more accurately than others.
Without putting specific values on them, here's a rough list of the main items in decreasing order of accuracy:
- Air pressure (highs and lows)
- Upper-level
- Surface
- Temperature
- Precipitation
- Areal coverage
- Amount
- Type (frozen or liquid)
Second, accuracy in general decays with time. The main reason is chaos theory. The atmosphere is one of those strange physical systems which can start from a given initial condition and evolve into multiple, different results. Theoretical calculations in the 1960s indicated that the limit of predictability for the atmosphere is somewhere around two weeks, and that estimate has not changed significantly in the meantime. (Note that we are talking about forecasting for a specific time at a specific location, not averages. In dealing with averages, it's a much different story, which is why a climate model is not at all the same as running a weather model for an extremely long time. Climate by definition is a long-term average, so even though they both start from the same base, these are two distinct kinds of problems, and in some ways climate prediction is an easier task.)
Besides the ultimate theoretical limit, there are practical limits, too. Even though the cell phone in your pocket is much more powerful than the supercomputers of a generation or two ago (more on that in a future post), there are still limits on computer speed and data capacity. Since the basic equations of the atmosphere can't be solved explicitly, they must be solved using approximations which require exponentially more time the more accurately they're represented. Using whatever technology is available at any given time, it's always possible to make a more accurate forecast, but it doesn't do much good to make a forecast that takes six hours of computer time, since by then a whole new set of data is available, and it's time to start the cycle all over again.
So, given the limitations, what's reasonable to expect? The chart shows a measure of the accuracy (skill) of model forecasts made by the European Centre for Medium-Range Weather Forecasting (ECMWF). The x-axis is the year, from 1980 through 2004. The y-axis is the number of days for which the forecast has some usable skill. The dashed line is the average for each month, and the solid line is a 12-month moving average. Note that the limit of skill goes from around 5.5 days at the beginning to around eight days at the end. Note also that the forecast is for the 500 mb pressure level of the atmosphere (around 5.5 km altitude, or about halfway up in terms of amount of air), which is, in general, at the top of the list of variables in terms of accuracy.
By Steve Scolnik |
February 28, 2008; 7:00 PM ET
Education
Previous: CommuteCast: Chilly Conditions Continue |
Next: Forecast: Windy Transition to March
Posted by: mcleaNed | February 28, 2008 7:06 PM
I'm going to turn this post to smut and PG-13 in a flash:
SNOW SNOW SNOW SNOW SNOW
HAHAHAHAHA, take that snow haters!
:P
Thanks for posting this!
Posted by: weatherdudeVA (Lake Ridge) | February 28, 2008 7:16 PM
mcleanNed,
Touche.
Tongue in cheek, of course.
Great piece on the models. I hope to see future essays with insights into the workings and subtleties of the computers and the forecasts that from them evolve.
Posted by: PJ Mt Vernon | February 28, 2008 7:24 PM
mcleanNed,
Touche.
Great piece on the models. I hope to see future essays with insights into the workings and subtleties of the computers and the forecasts that from them evolve.
Posted by: PJ Mt Vernon | February 28, 2008 7:24 PM
Ned,
Hahaha! Nice to see someone is following the theories.
Also nice to see some insight into the models. I hope to see more of these little tidbits in the future. Great work Steve!
Posted by: PJ Mt Vernon | February 28, 2008 7:33 PM
Much of what current inaccuracy there is in forecasting is that forecasters just depend on computer models too much. In this area, especially, there are two things that you simply have to KNOW from experience to make a good (or an acceptable) winter forecast.....and those two things, the models simply can't depict with reasonable acuracy. One is cold-air damming. Models often simply cannot pick up a situation where shallow, cold air with an E or NE wind is trapped east of the mountains in the lower couple of thousand feet....though it sometimes shows up on satellite photos when it is clear west of the Blue Ridge or west of the Allegheny front. This causes an overconfidence in bringing the SURFACE warm front too far north to agree with where the models forecast it......yet the front, ALOFT, is exactly where the models put it. But the models cannot pick up the surface cold-air damming because it is too small-scale an event in the boundary layer.
The second critical function in this area that models often cannot really forecast with any accuracy, is, of course, the rain-snow line, which, in some cases follows the 534 thickness line...others, the 540. even forecast experience, and having done it for years, sometimes doesn't help much with this. Though it often varies, the typical scenario for the immediate D.C. area, is wet snow changing to sleet and rain/freezing rain along a pattern from SE to NW, and then briefly back to wet snow again as the low center moves north.
Posted by: Mike | February 28, 2008 7:34 PM
snow
Posted by: missy | February 28, 2008 7:54 PM
Missy: I see you have the same dirty mind I do. LMAO!
Posted by: weatherdudeVA (Lake Ridge) | February 28, 2008 8:32 PM
I've lived in this area since 1954 & base a lot of my predicts on past history, & usually that is the typical snow-sleet-rain scenerio. I don't think enough attention is paid 2 past history when some forecasts come out. Some people just get 2 caught up in models, which almost always r wrong more than 2-3 days out. It seems like every week someone gets all excited about a model showing a snow storm that hardly ever pans out. Recently there was some excitment over a big snow storm on March 3/4, which isn't going to happen. Now there's some hope over a big storm a little later, at this point I just don't see it. In this area, it's best 2 expect the worse & hope 4 the best. As u get older you'll learn not 2 gets your hopes up. I hate 2 keep bursting everyones snow bubble, but snow in this area is just hard 2 come by.
Posted by: VaTechBob. | February 28, 2008 9:11 PM
Maybe winter was saving the best for last, meaning a major winter storm sometime in March. That would be a pleasant surprise.
Posted by: Yellow | February 28, 2008 9:19 PM
VaTechBob.,
Being a snow pessimist is a pretty easy job in VA. =P
Posted by: Period | February 28, 2008 10:20 PM
Is it wrong to wish for a major snow storm when those around you are praying for an early spring and telling you how crazy you are??
Posted by: Haymarket Dave | February 28, 2008 10:28 PM
Excellent post...Steve...which...BTW is something I/d like to see _more_ of on CWG.
SS wrote: "As good as the models have become, however, it's important to realize what they can and cannot do."
Which is supposedly why meteorologists have jobs in the first place...altho modern-day wx forecasting has evolved into little more than 'model reading.' We should all mourn the passing of the synoptician.
Paging Len Snellman.
Paging Len Snellman.
Len Snellman _please_ pick up.
Posted by: TQ | February 28, 2008 10:45 PM
I agree with the post for the most part. I think models need to be taken with a grain of salt. Especially this La Nina winter in particular.
On the FLIP side, part of the fun of this blog for many is checking the models when a storm is forecasted. Most bloggers know the unreliability of the models. Its part of the fun to hope this winter. I am sure if we had a big storm earlier in the winter, we wouldn't be so eager to model check...its just that winter is running out and these next few weeks are our last with some potential. I personally have enjoyed sharing my hopes of a winter storm...even though models have flopped all winter....we keep holdiing on...just what if.... Its the nature of the beast this winter with our snow drought.
Posted by: skinsfn | February 28, 2008 11:12 PM
00z models shifted the low a little east from 18z...
I had to say it..I didn't say it would snow. No hype here, just observing a change.
Posted by: Anonymous | February 28, 2008 11:26 PM
The problem with "snow" is that there's been so precious little of it this winter! It seems to be more accurate simply to forecast rain and crud for dance night!
Posted by: El Bombo | February 29, 2008 11:15 AM
The comments to this entry are closed.
A 2.5 day improvement in the last 28 years? I blame the butterflies. If they stopped flapping all over the place, there wouldn't be any chaos theory to begin with. :-)