The FAA Knowledge Test Mystery

Recently there were significant changes to the FAA Fundamental of Instruction knowledge test question bank (as well as the ATP and Flight Engineer). This change significantly affected the permanent FAA records of many who took this new test – the tests scores were significantly lower to include a record number of failures. The entire aviation community wanted to know why. Both the number and type of questions added to the test were significant. The release of these questions also came as a surprise to us as well. The questions that immediately came to my mind was “is this what the rest of the tests are going to be like?” and “what in the world are they trying to accomplish?” I was also concerned about the 300 plus students in Liberty’s aviation program; where these tests going to keep them in the chocks?

For those of you who use test prep software (e.g., ASA, Gleim, etc.) to prepare for the FAA written exam this may come as a surprise to you. The FAA does not release the questions to their tests; for some reason I was led to believe they did. The test prep software you use approximates the questions the FAA uses, but cannot completely replicate their tests. It should be no big deal though. As long as you study the source documents, then you will be fine. At least this is what I thought about the FOI test. From what my students tell me the questions are worded very differently; they are vague and confusing. There are two correct answers many times, etc. Well I went in search of the truth; it led me to OKC and the FAA Airman Testing Standards Branch.

I was invited, and attended the annual meeting at the FAA’s Airman Testing Standards Branch (AFS-630) in Oklahoma City on April 20th where industry and educational organizations were represented in a discussion about improving the testing process. Prior to this meeting the Society of Aviation and Flight Educators (SAFE) surveyed it’s members and forwarded the results of the survey to AFS-630 for their insight into some of the feelings about the test changes prior to this meeting. This survey asked the question, “Should the FAA notify the flying community of an upcoming change to its tests?” There was a mixed response but more importantly other concerns surfaced. Mainly they were about how the tests were constructed. The FAA took notice. Rather than go into details of the survey, I encourage you to read the results of their survey for yourself.

I was pleased with the discussion that occurred at this meeting where the FAA wanted to conduct further dialog with representatives from this meeting. The National Association of Flight Instructors (NAFI) and Aircraft Owners and Pilots Association (AOPA) both reported on the meeting so I won’t bother to repeat what they have already said. I encourage you to read the SAFE Survey report, NAFI Report, and AOPA Report to key on this important topic. Stay tuned for more words from the steering committee that is being formed to work with AFS-630. I consider myself very fortunate because I was asked if I would go back to OKC May 11th-12th and offer my experience in test writing – I feel woefully inadequate for this task, however, I guess my 6 years as an academic professor have given me some experience in constructing tests that meet a specific objective. I will report back as soon as possible regarding our work next week out in OKC. I welcome any inputs you may have as I take your voice to the FAA.

By the way, the individuals at AFS-630 are great people who are pilots and have a genuine interest in improving the safety record of GA pilots. The changes to the test question bank were an effort toward those ends. I agree with their objective, I just don’t agree with their method. I also am glad that they are listening and actively engaging this issue to make it better. Let’s hope for the best in this effort.

Leave a Reply