Life is an Open-Book Test

I knew what the e-mail was going to be about the minute I saw the sender and the title: My Final Grade for the Semester. I dreaded the necessary response. The message came from Sarah, one of the top students in my recent graduate class in security concepts and policy. I had formally submitted the final grades for the semester two days earlier. After scoring in the mid-90s on all her previous examinations and quizzes, she posted a very poor grade on the final that dragged her grade down to a B after the semester scores were tallied and averaged. She had wanted an A.

Like any driven student with lofty goals, she was aiming for a “straight-A” master’s program. I guessed correctly that mine was the first B to grace her end-of-semester report card. I truly felt sorry for her, but there was really nothing I could do that would satisfy her. When I first started teaching graduate school many years ago, I had made a commitment never to negotiate a student’s grade after the end of the semester.

Throughout the semester, I take whatever time is necessary inside or outside class to help each student achieve their goals. After the final is submitted, though, one has to close that window of opportunity. I wish I had a dollar for every student who has endeavored to argue over a final grade after the course is over and all graded materials have been submitted. I know if I were to roll over for one, I’d have to do it for everyone.

Many of my students are initially pleasantly surprised when they find all my tests are open-book and open-note examinations. That is, until they see the questions. That’s when they realize the tests aren’t comprised of quick look-up, multiple-choice queries. I ask questions that require students to assimilate information they have gathered during the lectures, outside reading and exercises. They then must be able to express the answer in the form of two to five paragraphs of prose, depending on the depth of the question.

For the final in this course, I put forth a scenario for the students, and ask them to role-play. For example, I’ll say they are at a job interview for a senior security director or analyst. I then pose questions one could expect from a hiring manager. I ask them to respond appropriately using information they learned in class. In order to prepare the students for this exam, I make sure we practice role-playing for at least two class sessions.

This was the part of the course where Sarah struggled. Until this time, she was easily able to spout off answers to every question I asked in class or on tests. She would deftly flip to the section of her notes or a chapter in one of the books to give an accurate response; however, when it came time to wrestle with a real-world situation, she had a difficult time. Knowing information is important, but knowing how and when to apply it is critical.

The reason I employ open-book, open-note exams is simple: our work life is an open-book test. In graduate education, I feel security educators should be looking past rote memorization and the regurgitation of facts. In an educational setting, it becomes more important to show students (many working adults) how to combine the knowledge they learn in class with their professional experiences to apply to real-world situations.

Sadly, too many educational and professional examinations are designed to simply measure basic retained knowledge. This, of course, makes it easy for the examiners. They produce a large volume of questions with multiple choice answers, rotate these questions in and out of the exam, and grade the little penciled responses with a computer. It’s quick, easy, and they can charge a handsome fee for this service to “certify,” or pass the successful respondents.

This process may be adequate to document someone’s basic retained knowledge, but it does little to determine how effective they will be as an employee or entrepreneur. For security professionals, it’s not simply a matter of understanding the best video technology or knowing distinctions among network protocols — these all change. It is vital, however, to know where to find out the specifics of these issues, and how to best implement them within organizational policy and fiscal constraints.

I have the luxury of teaching at an enlightened department in a world-class university. I perform my duties with the goal of making a positive impact on the professional lives of my students. I realize I perform a different function than certifying bodies and testing services; however, your ultimate success as a security expert will not reside in the number of certifications you can acquire. Your victories will ultimately be defined by the people and resources you protect from those who seek to exploit them. That challenge is an open-book test.

John McCumber is a security and risk professional, and is the author of “Assessing and Managing Security Risk in IT Systems: A Structured Methodology,” from Auerbach Publications. If you have a comment or question for him, please e-mail John at: [email protected].