Security Implications of Microsoft® Windows Vista™

Introduction

Windows Vista is the result of over four years of work and the investment of many billions of dollars. It is billed as the most secure version yet of the Microsoft Windows® operating system. This paper discusses not only the security technologies employed by Microsoft that justify this accolade but also how, in combination, these technologies mitigate specific classes of threats. This paper presents a high-level summary of Symantec’s research findings into the security of Windows Vista, and a set of conclusions that discuss the exposure that remains even in the face of its new security technologies. The intent of this paper is not to detract from the improvements that Microsoft has made, but rather to provide an objective and balanced view of how Windows Vista will affect the overall threat landscape.

Symantec started researching Windows Vista in 2005 and has monitored its development carefully. The goal of this research has been to understand the technology improvements being made by Microsoft and also to understand the threats facing the new operating system and, in turn, Symantec’s customers.

Security technologies in Windows Vista

With the introduction of Windows Vista, Microsoft has leveraged a number of security technologies in order to mitigate several classes of attack that have historically plagued the Windows operating system.

These technologies are numerous, and are best depicted visually.

These technologies can be broken down into three core categories:

• Generic exploit mitigation

• Kernel integrity

Generic exploit mitigation

This category of mitigation is designed to prevent attackers from successfully exploiting applications that contain specific classes of code-level vulnerabilities. The technologies employed here fall into two key categories: developer-controlled and operating system improvements. When combined, these techniques successfully inhibit the exploitation of memory corruption and memory manipulation vulnerabilities. This includes the following common classes of software flaws:

• Stack buffer overflow vulnerabilities

• Stack function pointer overwrites

• Structured exception handler overwrites

• Heap overflow and structure manipulation

The technologies introduced in Windows Vista are very effective at protecting the core Windows operating system as well as Microsoft compiled applications. They serve to make the exploitation of traditional vulnerabilities infeasible, including those leveraged by well-known widespread worms observed earlier this decade. As a result, the overall impact of some code-level flaws, even when introduced by a Microsoft software engineer, is greatly diminished.

Developer-controlled technologies

Developer-controlled technologies can be leveraged by software engineers in order to make their applications more robust. These technologies can be incorporated either through the enabling of compiler options or through the introduction of explicit code changes.

The technologies that fall into this category are:

• Pointer obfuscation

•GS

• Safe Structured Exception Handlers (SafeSEH)

• Address Space Layout Randomization (ASLR)

• Terminate on Heap Corruption

Analysis of developer-controlled technologies

One barrier to the success of these technologies is the requirement for third-party software vendors to explicitly leverage them. Software engineers must utilize the latest version of Microsoft’s development tools in a specific manner. Only by doing so can they enable the functionality that is designed to inhibit or minimize the impact of the different exploitation techniques.

Only when developers recompile their application or, in certain instances such as pointer obfuscation, make modifications to their application’s source code will they benefit from these improvements.

While the majority of newer Microsoft applications are expected to use these technologies, older sof tware and software written by third parties may not. As a result, older Microsoft or third-party applications and drivers will continue to pose a risk, as they will remain largely unprotected. This fact has already been borne out with the recent announcement of vulnerabilities present in the Windows Vista version of a common server application.

Symantec researchers noted that in some cases even core Windows Vista components failed to adequately leverage these technologies. Specifically, a small percentage of Windows Vista 32-bit has not been compiled with GS technology from Microsoft Visual Studio® 2005.

The reason for the exclusion of these applications from the protection afforded by this technology is unclear. It is acknowledged that these components pose a greater risk than those that are protected.

Consequently, these components of Windows Vista are not protected against the aforementioned class of memory corruption and memory manipulation vulnerabilities. While the exposure to risk resulting from this circumstance is low, it does serve to increase the potential attack surface for Windows Vista. Symantec expects attackers to identify these vulnerable points and investigate their potential.

Operating system improvements Operating system improvements are technologies that are native to the core operating system. While similar in overall effect to developer-controlled technologies, their function is ultimately implemented by components within the core operating system. The technologies that fall under operating system improvements are:

• Heap manager improvements

• Data Execution Prevention (DEP)

• Safe Structured Exception Handlers (SafeSEH)

• Address Space Layout Randomization (ASLR)

• Terminate on Heap Corruption

Analysis of operating system improvements

Like those discussed in the previous section, the majority of technologies falling into this category also require that software engineers first enable them in their application. Of these four different technologies, only the first (heap manager) applies by default to the operating system as a whole. The second (DEP) is enabled only for Windows Vista core operating system components and not for some common applications such as Internet Explorer. The final three require developers to specifically enable support in their application during development.

As a result, third-party applications, as well as those developed by Microsoft that are not considered part of the core operating system, are not afforded equal protection even with the introduction of these technologies.

Limited scope of Data Execution Prevention

In default installations of Windows Vista, Symantec observed that one technology (DEP) is applied by default only to the core operating system.

Default DEP configuration

This limitation leaves third-party applications on Windows Vista with less protection than the core Windows Vista operating system and service. This fact increases the likelihood of successful exploitation of vulnerabilities present in these applications. As mentioned previously, even common applications such as Internet Explorer do not leverage the benefits of DEP.

ASLR: Not as random as expected Symantec performed an in-depth analysis on the effectiveness of Address Space Layout Randomization (ASLR). The purpose of this technology is to randomly locate programs in memory and, by doing so, enhance security. This enhancement comes from the attacker’s inability to know exactly what to target during the exploitation of a vulnerable program. When implemented correctly, this technology is extremely effective in mitigating the exploitation of memory corruption and memory manipulation vulnerabilities.

The results of this analysis show that at least one aspect of ASLR’s implementation did not perform as expected. Symantec found that one of the randomized components was not randomized consistently, resulting in a reduced degree of randomness in the layout of an application’s memory. While ASLR continues to be effective, this reduction does increase the likelihood that an attacker can guess the correct address to target.

Microsoft has confirmed Symantec’s research findings and resolved the issue highlighted. These shortcomings are due to be addressed in Windows Vista SP1.

Kernel integrity

The kernel is the core component of any modern operating system. It is the central building block upon which the security of the system is built. Should the kernel be compromised or subverted in any way, then the underlying foundation can no longer be trusted. Kernel integrity and security have become a hot topic in recent years due to the aggressive evolution of rootkit technologies. These technologies are used by attackers and threats to hide their presence while also providing potential backdoors into the system. In addition the evolution of Digital Rights Management (DRM) provides another, arguably even stronger

incentive for securing the kernel to avoid the unauthorized interception of audio and video content.

For this reason Microsoft has invested heavily in technologies that can help improve the reliability and security of the Windows Vista kernel. The three technologies employed by Microsoft to improve kernel

security are:

• Driver signing

• Code Integrity

• PatchGuard

Driver signing is designed to ensure that all kernel drivers loaded by the system are signed by a trusted authority. The goal of this technology is to ensure that only code that has been tested by Microsoft or signed by a trusted developer is loaded into the kernel—with the side effect of stopping malicious code from loading into the heart of the operating system.

Code Integrity is designed to ensure that the core operating system has not been tampered with either accidentally or maliciously. Code integrity verifies the digital signature and associated hash on core operating system binaries (in particular kernel components) in order to detect this tampering.

PatchGuard is the most controversial of these technologies. Whereas Code Integrity protects core operating system files on disk and in memory, PatchGuard protects key operating system structures from being patched or extended in kernel memory. Vendors such as Symantec have historically used this patching technique to provide protection at the lowest level possible to ensure the maximum protection against malicious code such as rootkits. However, these same techniques are utilized by rootkit writers to ensure the stealthiest operation possible.

Analysis of kernel integrity technologies

It is important to note that only the 64-bit version of Windows Vista benefits from this category of technology, while 32-bit Windows Vista, expected to be the standard deployment for years to come, does not.

As demonstrated during the development process of Windows Vista and during its release, hackers can and will subvert PatchGuard.2 The kernel integrity protection mechanisms that are present on 64-bit Windows Vista can only be described as a bump in the road. That is, while these technologies may slow down an attacker, they may not provide a meaningful defense against a determined one.

Symantec researchers investigated the feasibility of disabling all three key kernel integrity technologies:

driver signing, Code Integrity, and PatchGuard. Results have shown that all three technologies can be permanently disabled and removed from Windows Vista after approximately one man-week of effort.

A potential victim need make only one mistake to become infected by a threat that does the same.

The result: All new security technologies are stripped from Windows Vista in their entirety.

System integrity and user-mode defenses

Microsoft’s system integrity and user-mode defenses are numerous, and their purpose is clear. Microsof t’s strategy is to run software with the minimum set of privileges required and, where possible, to run applications in a compartmentalized environment. This approach is further strengthened by reliance on signing to provide assurances about the identity of the publisher of software. Such assurances allow the user to make informed decisions about running an application and allowing it to perform actions on the host when prompted.

The goal of these technologies is to encourage users to run programs at a reduced privilege rather than running everything as Administrator, forcing them to consider the consequences of their actions. In addition, these technologies seek to reduce the ability of malicious code to automatically compromise the entire system.