Wx Watch: Flunking the Wx Test: Test Results

Back in AOPA Pilot's January 2000 installment of "Wx Watch," we talked about some of the problem areas surrounding the issue of weather education. The central thrust of the piece (see " Wx Watch: Flunking the Weather Test," January Pilot) was that, in light of accident statistics, our current methods of teaching aviation weather seem to be inadequate in that they don't prepare us for real-world weather encounters. The weather questions on the pilot certification knowledge tests in particular came under fire for being little more than exercises in rote memorization. The multiple-choice questions dealing with weather issues were cited as being too few, too simplistic, and too lenient in that they were more a test of the candidate's test-taking abilities than his or her weather savvy.

Many readers responded to the article, either by e-mailing me or making submissions to a Web address on AOPA Online dedicated to member sharing of views on the subject. Judging by the response, it appears that the problem of weather education has hit a nerve. And the test questions weren't the only issues foremost on the list of pilot concerns. It seems that everyone has a gripe about the way weather is presented, represented, or taught, and readers used their article responses as vehicles for airing their grievances.

Some members sounded a confessional note, admitting that their weather knowledge was extremely limited.

"Although I'm a 1,500-hour IFR, twin-rated pilot, I still feel inadequate interpreting basic weather phenomenon," said one respondent. Another said, "Although I've been flying for about six years, it's only over the last two years or so that I have really started doing long cross-countries, most of them in IFR. I have to tell you that I am shocked at how much I do not know about how weather works. The more I fly in this environment, the less I seem to know...." The same writer went on to say, "It seems difficult to test someone's weather savvy through a written test. What you really want to know is whether someone can combine theoretical/scientific knowledge with sound decision making to arrive at a reasonable decision regarding a particular weather situation." Another pilot said, "I worked very hard to ace the written and practical...but I still feel inadequate in flying...I've fallen into the habit of flying only in the best of conditions and shunning even overcast days with 3,000- and 4,000-foot ceilings."

Flight instructor education is another problem, some felt. One writer summed it up by saying, "One factor of this issue that really needs to be addressed is flight instructor education — hell, I have known some CFIs and CFIIs that just teach to the tests!"

One pilot even named what he felt was the most useless weather question on the private pilot knowledge test: "‘Which weather phenomenon is always associated with a thunderstorm?' Why does this win the award?" the writer asked. "Some pilot out there...will come up with the following logic: There have been no reports of lightning. No lightning, no thunderstorms. So, let's go flying!"

Others took the opportunity to take a swipe at flight service stations. "Today the system seems to be controlled by CYA...as a result, I seldom listen to the warnings and recommendations of briefers, but listen to the weather and make my own decision," said one member. "Most briefers do nothing but report what they think are the important elements, and usually leave out the important features of the approaching weather...we have a local TV weather reporter who can do more in reporting the national weather scene in five minutes than most briefers can do in a half-hour of punching on a computer," railed another.

A contrary view stressed the value of FSS briefers: "If it were not for human weather briefers and it was solely left to me to interpret the weather, I hate to admit it, but I'd be in real trouble interpreting all the data in its present format. It is absolutely not user-friendly and leaves you thinking it's some kind of secret code...."

Interpreting codes doesn't bother other pilots, as summed up by another writer: "Every GA airport should be required to have a computer available for GA pilots to access at least DUATS." With DUATS and many other weather providers, plain-English weather reports and forecasts can be ordered up with just a couple of keystrokes.

One respondent went so far as to say, "We ask pilots to learn too much weather they will never see, with the result that they quickly forget essentials." This was not the majority view.

Another blames poor ASOS (the automated surface observing system) reports for misleading pilots. "The most recent assessment by the Air Force shows ASOS reporting ceilings too high during MVFR and IFR. It also shows that during periods of lowering visibility it lags," asserted one pilot.

Many had harsh words for the philosophy behind the FAA's knowledge exams. "Weather is a spatial process that occurs over time, and we test our understanding with linear (or pointal) thinking...it's too bad that when we look at the sky, we can't see the colored arrows and plastic overlays," said one writer. There was almost universal agreement that the knowledge tests should be changed to a more substantive format.

One reader suggested borrowing airline standards to reduce general aviation's weather accidents. "An insurance broker for airline pilots," he said, "reports that the second most frequent cause of death for airline pilots prior to age 60 is a general aviation accident. Even worse, the cause of GA accidents involving professional pilots exhibits the same bad decision making as the GA statistics. When I examined the difference between these two worlds, I realized that there were two significant differences in the decision-making process: 1) Most airline weather decisions were not made in the cockpit but on the ground. Airline operations specifications and manuals and pilot handbooks set objective criteria for most weather decisions; and 2) A system of accountability reinforces objective decision making. There are layers of responsibility: first officer (CRM boosted his/her role), dispatcher, FAA, fellow pilots in other aircraft, to name a few.... The real problem in GA decision making may not be knowledge but the process. Flights that ought to be taken are not begun, and flights that shouldn't are.... I sat down and wrote an Ops Spec for my personal flying. The results are better, more relaxed, and consistent decisions."

It wasn't all one huge gripe session, however. Many, many writers offered up constructive suggestions on how to improve the ways pilots learn aviation weather. Here are a few:

• "VFR pilots do not understand the dangers of vertigo from lack of training in IMC.... Early in my instrument training my instructor scheduled a flight in IMC...and I'm forever thankful for this lesson."
• "Student pilots should have at least 10 hours of instrument training...with three of those hours in actual IMC with instruction on making 180-degree turns and getting back to VFR."
• "A minimum of 25 hours of flying solo [should be required] after receiving your private certificate, before taking passengers."
• "Emphasize the importance of understanding the 'big picture' of pressure systems and fronts before getting into the detail of reports and forecasts."
• "Include more real-weather flying in pilot training."
• "Make weather analysis and reporting a part of pilot training. Student pilots should be made to gather the information themselves, and then forecast their own local weather. Later, they can check the validity of their own predictions."
• "AOPA should collaborate with a university that specializes in ISD (instructional systems design), the way the U.S. Air Force and others apply a systematic approach to applying educational dollars and theories to develop an educational CD-ROM, video, workbook, etc. The point behind ISD is to clearly identify exactly what the problem is, then design the correct fix."
• "If they insist on the multiple-choice format, I would at least like more questions requiring the applicant to look at several charts/forecasts/reports and determine which of three scenarios might be encountered."
• "How about better simulators that allow pilots to experience weather before they actually get into it?"
• "The new glass cockpits of the future with datalinks and real-time weather display with visibility and ceiling information will help pilots steer around or land when the picture shows 1,000-and-three or less."

While we don't need a write-in campaign to prove that our weather education policies aren't up to par, it was gratifying nonetheless to observe the level of reader participation and intellectual investment in the issue. Even Dr. Irvin Gleim, the publisher of the very popular Gleim test preparation books, agreed that the FAA tests could be better. "I wholeheartedly agree that the FAA knowledge tests should foster learning and understanding rather than be a memorization marathon. I believe the success of our Red Books is due to their ability to provide more than the answers.... We attempt to explain the subject matter so as to develop pilots' analysis, synthesis, and evaluation abilities."

Many thanks to all who shared their views. In the future, look for more opportunities to express your ideas on weather issues. As the saying goes: Everybody talks about the weather, but no one does a thing about it. Maybe we can't will ceilings to rise, but we certainly can raise pilot awareness by providing a forum for discussion. Hopefully, the players in the weather education business are reading.