Data Protection and Security

VI

Operating System Security and Secure Programming

VI.III

Security Evaluation

Case Study:

Microsoft sponsored an evaluation of Windows 2000 (with Service Pack 3 and one patch) against the Controlled Access Protection Profile (plus some enhancements) and obtained an EAL4 evaluation rating. This is most accurately written as "CAPP/EAL4".

Though this looks surprising and impressive at a first glance, a more careful treatment identifies two important problems in this evaluation study:

Problem 1: The Protection Profile

The Controlled Access Protection Profile (CAPP) standard document can be found at the Common Criteria website. Here is a description of the CAPP requirements taken from the document:

“The CAPP provides for a level of protection which is appropriate for an assumed non-hostile and well-managed user community requiring protection against threats of inadvertent or casual attempts to breach the system security. The profile is not intended to be applicable to circumstances in which protection is required against determined attempts by hostile and well funded attackers to breach system security. The CAPP does not fully address the threats posed by malicious system development or administrative personnel.”

If we translate this into colloquial English:

“Don't hook this to the internet, don't run email, don't install software unless you can 100% trust the developer, and if anybody who works for you turns out to be out to get you, you are toast.”

Problem 2: The Evaluation Assurance Level

An EAL4 rating means that you did a lot of paperwork related to the software process, but says absolutely nothing about the quality of the software itself. There are no quantifiable measurements made of the software, and essentially none of the code is inspected. Buying software with an EAL4 rating is kind of like buying a home without a home inspection, only more risky.

In summary, this case study shows that evaluation by an independent party does not eliminate the need for expertise in order to choose or compare security systems. Understanding what the security evaluation means also needs background.

Reference: The Controlled Access Protection Profile (CAPP)

More to Read: EAL4