AOPA will be closed Monday, January 20th in observance of the holiday. We will reopen Tuesday morning, January 21st at 8:30am ET.
Get extra lift from AOPA. Start your free membership trial today! Click here

Checkride

The law of percentages

How the FAA grades DPEs

A common question student pilots ask a designated pilot examiner (DPE) is, "What percentage of the pilots you test pass their checkrides?" It's a fair question, and the answer is also important to the FAA as it fulfills its responsibility to assess and maintain the highest standards of quality among the DPEs the agency oversees.

By government decree, the FAA has the authority and responsibility to supervise and maintain the quality of the work accomplished by its designees. These standards and requirements are specifically addressed in FAA documents Order 8900.1, the Flight Standards Information Management System (FSIMS), and Order 8900.2, the General Aviation Airman Designee Handbook.

Of the several methods available, observing the DPE actually conducting a practical test is the FAA's preferred method of evaluating a DPE's effectiveness. And just as any pilot taking a practical test must perform specific tasks within the tolerance limits specified in the practical test standards (PTS), every DPE also must meet a similar list of performance standards in order to continue issuing certificates.

Besides being a complete source for all the details associated with becoming, maintaining, renewing, and operating as a DPE, these documents also specify that in addition to the annual requirement of being observed while conducting a complete practical test, several additional circumstances might also require additional FAA surveillance inspections during the year. These include issues like conducting more than two practical tests per day, having a filing error rate that exceeds 10 percent during the year, being involved in an accident or incident, or committing a violation of the regulations. Topping that list is an examiner whose practical test-passing rate exceeds 90 percent. Likewise, it should also be noted that CFIs having a student failure rate that reaches 30 percent or greater are likely candidates for a similar visit with the FAA.

Although the relative difficulty of any practical test is largely determined not by the DPE, but by the FAA through the PTS established for that test, one might conclude that DPEs would feel pressure to increase the difficulty of their practical tests in order to stay out of the danger zone. But considering that the national pass rate for private pilot practical tests over the past several years has hovered between 76 and 77 percent, it would appear that few DPEs would need to toughen their tests. On the other hand, the FAA also directs that examiners must not allow the conduct of successive practical tests to degenerate into a predictable pattern that can be recognized by applicants or instructors. DPEs generally accomplish this requirement by varying the way in which questions or flight-task scenarios are presented to the applicants. In this way, while creating a more difficult test is not the intended goal, through the process of maintaining test effectiveness the perception of creating a more challenging test could possibly result.

You might also be thinking that when a DPE has conducted several successive passing checkrides, that examiner might sense some pressure to produce a failure in order to keep his average in balance. This is not the case. In my own experience and from conversations with other DPEs over the years, examiners genuinely love to see applicants perform well on their checkrides, and the high point of every DPE's job is issuing successful applicants a new certificate. But unfortunately for some applicants, the law of percentages somehow seems to work against them and a DPE never has to wait very long for a failure to crop up after a string of passes, thus keeping his pass/fail ratios steadily well below the FAA's 90-percent limit. Indeed, the national average pass/fail ratio suggests that the 90-percent threshold set by the FAA creates an appropriately sized and useful yardstick in measuring the effectiveness of the practical tests conducted by individual DPEs.

Almost universally, DPEs begin each test as a completely clean slate, making a determination that a failure has occurred only after the applicant has displayed an area or areas of knowledge or skill that have fallen clearly below the stated PTS tolerances. DPEs will almost always give the applicant the benefit of the doubt when he first stubs his toe, and will not throw in the towel at the first sign of trouble. Even the FAA has given its DPEs some latitude in these situations by stressing that if the DPE is in doubt, he or she should continue testing the applicant until certain that a notice of disapproval is indeed called for.

In addition, while failure to meet the tolerances outlined in the PTS for any one required task is grounds for failure, the PTS also lists a variety of additional reasons for failure that includes a "consistent" failure to meet PTS tolerances. This implies that a DPE is not forced to fail an applicant after the very first hiccup, but has some subjective latitude to continue testing to determine if it was just a single hiccup or a symptom of an acute deficiency requiring additional training from the CFI. In any case, the DPE's role throughout the test is a pilot knowledge and skills evaluator, making the DPE simply a witness to and not the reason for the demonstrated performance of each required task.

So as your test date draws near, it's only natural to want to know what your chances of success are as you ask your DPE the common question, "What percentage of the pilots you test pass their checkrides?" But when you stop to think about it, there is really only one answer that truly matters - 100 percent of applicants properly prepared for their checkrides pass, and 100 percent of applicants who arrive unprepared will fail.

Bob Schmelzer is a Chicago-area designated pilot examiner and a United Airlines Boeing 777 captain/line check airman. He has been a flight instructor since 1972.

Related Articles