I recently returned from OKC where I was part of a group of educators and industry personnel who met with personnel in AFS-630 (Airman Testing Standards Branch) for two days. Those present were representatives from AFS-630, AOPA, NAFI, Liberty University, Embry-Riddle Aeronautical University, Western Michigan University, University of Oklahoma, and the Professional Aviation Board of Certification. The reason for this meeting was to discover the reason of a sudden, and drastic drop in the pass rates of the Fundamentals of Instruction (FOI) examination.
During day one, the group reviewed each of the new questions introduced into the FOI question database on February 14, 2011. The thinking was that there must be something wrong with these questions; they needed immediate attention. As it turns out, only eighteen of the new questions were identified as needing some type of revision. Of the eighteen questions, six were identified as requiring immediate attention. The problems with the other twelve questions generally related to inaccuracies in the source document, the Aviation Instructors Handbook. For these questions, one of two approaches was suggested; 1) change the question so that it aligned with the source document, or 2) eliminate the question until the source document could be fixed. Option #1 seemed to be the more appropriate thing to do since the issues were minor in the big picture and the fact that changing a source document would take significantly more time and effort to achieve. The long-term option of correcting the source document, and the related questions, was also recommended to eliminate confusion among the pilot population. The bottom line take away from day one was that applicants needed to pay more attention to the source document for the FOI test, the Aviation Instructors Handbook. There was a sense that applicants were relying on other methods of test preparation in lieu of reading the actual textbook the test is taken from. At the end of the day, the six questions identified as needing immediate attention were removed from the question bank. The good news is that those individuals affected by these defective questions will receive credit and their airman records will reflect accordingly. Stanley Roberts, Manager of AFS-630 stated the he would personally call each of these individuals to inform them of the good news. I credit not only the work of the group of individuals, both FAA and industry, but also the leadership in AFS-630 (Stanley Roberts) and AFS-600 (Van Kerns) – my hats off to them!
On day two, the discussion about where we go from here was addressed. A conference call with Kent Lovelace, University of North Dakota Department of Aviation Chair and NBAA Certified Aviation Manager (CAM) Governing Board, Organizational Director, provided insight into how the CAM Governing Board handles their testing process for the certification of Corporate Flight Department Mangers. He is directly involved in that process and has a wealth of knowledge and experience in this area. It was enlightening to hear about how other organizations handle this testing process. It was determined, by Van Kerns (AFS-600), that a working group be established to look at ways to improve the testing process. The initial focus will be to gather education and industry experts in the testing discipline to work with the FAA to create an evaluation process that would best serve the aviation profession, which will lead to a safer, more prepared applicant. After that, sub-groups will more than likely be formed to consist of personnel with more specialized skill sets so that each area of expertise is evaluated fairly. I am excited about these changes and feel that this process will produce better aviation professional; the result will be a safer aviation system.
Others reporting on this two-day visit are AOPA, NAFI, as well as an official AFS-630 notification addressed to the attendees of the original meeting held in mid-April – the following is the content of that notification:
“In light of concerns raised about recent changes to the FAA’s Fundamentals of Instruction (FOI) knowledge test, which is a prerequisite for instructor certification, the Regulatory Support Division of the FAA Flight Standards Service (AFS-600) consulted with academics from Embry-Riddle Aeronautical University, Liberty University, the University of North Dakota, the University of Oklahoma, Western Michigan University, and others with expertise in the area of test writing and validation. As a result of these discussions, six questions are being withdrawn immediately from the FOI knowledge test. Test scores for flight instructor candidates whose FOI test included any of the six questions will be adjusted accordingly.
The FOI exam was updated to test items added or revised with the 2008 edition of the FAA Aviation Instructor’s Handbook (FAA-H-8083-9A). Because this edition incorporates a number of changes and updates, the FAA stresses that instructor candidates should use it as the primary text when preparing for the FOI test.
The Flight Standards Service’s Regulatory Support Division promotes aviation safety through validation of airman knowledge, and strives for continuous improvements in the knowledge testing process. To that end, the FAA intends to involve the flight training industry and academia in the process of developing future revisions to testing and guidance documents. The FAA is also developing more detailed knowledge test guides to assist airmen and aviation instructors in preparing for knowledge exams.”
Stay tuned for more updates.