Data Protection and Security

VI

Operating System Security and Secure Programming

VI.V

Buffer Overflow Attacks

At the end of this chapter, we look at the secure software problem from the reverse angle or in other words from the attacker’s point of view and we briefly explain how buffer overflow attacks, which is one of the most common software vulnerability in CERT advisories, can be exploited.

Computer programs generate sections in memory for data storage. When contiguous chunks of the same data type are allocated, the memory region is known as a buffer. If you try to put more data in a buffer than it fits, the extra data have to go somewhere and the next contiguous chunk of memory is overwritten. This is called buffer overflow. Languages like C and C++ are inherently unsafe because there are no runtime checks that prevent writing past the end of a buffer.

Buffer overflows are not always security problem (i.e. programs can fail completely). However sometimes the modifications on the meaningful data the overflow has caused might lead to security problems. One way in which buffer overflows cause a security problem is through stack-smashing attacks.

The stack-smashing attacks first prepare some attack code and fill the buffer with it and then overwrite the stack in such a way that control gets passed to the attack code This usually requires overwriting the return address to redirect the execution to either somewhere in the buffer, or to some library function that will return control to the buffer. e.g. The attack code is followed by the address of the beginning of the code which should overwrite the return address on the stack.

By exploiting this attack, it is possible to write malicious software; either a worm which is a self-replicating self-contained program or a virus which spreads by attaching itself to other programs.