The aim of this page, which may be updated during the semester, is to give you some hints for the CTF. Remember that each team has to manage its own virtual machine, running some vulnerable services on a Gentoo Linux operating system, hence you may need to know some useful commands to list or kill processes currently in execution, find out active connections on your machine, connect to a MySQL database via command line and so on.
Furthermore, during the CTF you should develop some scripts to avoid to perform manually the attacks and the submission of the flags during each round, so we will show you how to perform easily HTTP requests using Python.
Clearly this cannot be a comprehensive guide: if you need more information, please refer to the manual page of the related command or ask Google 😀
If you think that something really important is missing or you haven’t understood something, feel free to post a comment!
Change your password
The first thing you have to do when the competition starts is to change the root password of your virtual machine: since you will connect directly as root with ssh, it is enough to use the command passwd.
root@diff:~# passwd
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Switch user
In your machine there will be a different user for each service: since it is a good habit to use root only when strictly necessary, you may consider the possibility of switching to the user that owns the service on which you want to work. This can be done with the command su - ; if you want to go back to the previous user, you can use the combination CTRL + D or the command exit.
root@diff:~# su - pin
pin@diff:~$ id
uid=1001(pin) gid=1001(pin) groups=1001(pin)
pin@diff:~$ exit
logout
root@diff:~#
The manual
If you need to know more information about a command, you must simply type man <command_name> on the command line. You can search for a word typing /<word> once the manual page is open and, if you press n, the next occurrence will be shown. You can move inside the page using arrow keys. When you have finished, type q to exit.
You can also get the documentation of many functions of the standard C library using man 3 <function_name>.
Do you want to know more about man? Read the manual of the manual 🙂
Files search
find is a very powerful tool for searching files inside a given path with some particular properties: for instance you can ask to list files that belong to a given user, search for files on which you have particular permissions and so on. You can also combine different properties using logical operators.
For instance, this command allows you to find regular files (not directories) owned by the specified user that are readable by everybody:
diff@diff:~$ find /home/pin/ -user pin -perm /a+r -type f
/home/pin/challenge/pin.py
/home/pin/challenge/server.py
/home/pin/challenge/pin.pyc
Instead, if you want to search, only in the working directory, for files whose names start with .b or with permissions exactly 755, we can use:
diff@diff:~$ find . -maxdepth 1 -name '\.b*' -o -perm 755
.
./.bash_history
./.bash_logout
./.bashrc
Refer to the manual if you want to know more about this utility.
Process management
A common way to obtain complete information about the processes currently in execution on the system consists on using ps with options aux. Here is an example of (part of) the result produced by the command:
diff@diff:~$ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.3 3504 1764 ? Ss Feb11 0:02 /sbin/init
root 2 0.0 0.0 0 0 ? S Feb11 0:01 [kthreadd]
diff 238 0.0 1.6 14808 8444 ? S Feb11 0:01 /usr/bin/python /home/diff/challenge/server.py
diff 17417 0.1 0.2 4936 1232 pts/2 T 18:59 0:00 nano
diff 17418 0.0 0.2 5204 1192 pts/2 R+ 18:59 0:00 ps aux
diff 32098 0.0 1.2 12400 6256 pts/4 S+ Apr17 7:44 python -m SimpleHTTPServer
The fields to which you may be interested are probably USER, PID and COMMAND that respectively denote, for each process, the user that has started it, its process identifier and the command given to start it.
You can kill a process using the command kill and specifying the PID: for instance, if we want to kill nano:
diff@diff:~$ kill -9 17417
diff@diff:~$ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.3 3504 1764 ? Ss Feb11 0:02 /sbin/init
root 2 0.0 0.0 0 0 ? S Feb11 0:01 [kthreadd]
diff 238 0.0 1.6 14808 8444 ? S Feb11 0:01 /usr/bin/python /home/diff/challenge/server.py
diff 17425 0.0 0.2 5204 1192 pts/2 R+ 19:00 0:00 ps aux
diff 32098 0.0 1.2 12400 6256 pts/4 S+ Apr17 7:44 python -m SimpleHTTPServer
If you want to kill processes that are not owned by your current user, you have to switch to the one that owns the process (or to root).
You can even specify to kill processes with a (partially) given name or that belong to some specified users and so on using pkill: please refer to the manual for more information.
Network connections
Information about active network connections can be seen using netstat as follows:
diff@diff:~$ netstat -natup
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8000 0.0.0.0:* LISTEN 32098/python
tcp 0 0 0.0.0.0:42765 0.0.0.0:* LISTEN 238/python
tcp 0 0 0.0.0.0:42766 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 672 192.168.69.22:22 192.168.69.159:51987 ESTABLISHED -
tcp6 0 0 :::22 :::* LISTEN -
Column Proto reports the transport protocol in use, Local Address and Foreign Address denote, respectively, addresses of the parts involved in the connection, State is the state of the connection (except for connectionless protocols) and PID/Program name shows the PID and the name of the program that has created the connection. The last column shows information only of processes owned by the user that gives the command: if you want to see them all, you have to run the command as root.
For instance, the first row says that there is a Python program listening on the port 8000, while the fifth says that there is a connection on the interface with IP address 192.168.69.22 on the port 22 with a machine with address 192.168.69.159 on the port 51987.
As we have seen during the course, you can quickly open a connection with an entity using nc (netcat), specifying the IP address (or the domain name) and the port. An example follows, referred to a service of the previous year:
diff@diff:~$ nc 192.168.69.22 42765
Insert your 5 digits user ID (trailing zeros allowed) for the generation of the new password: 12345
New password: nz41HgDXQ7
Data to be stored in the key: 750f286135dc1dc745d417a2482a225b219b96c2d6401b2b
Apart from this, you can also use nc to listen for a connection and so on: if you want to know more, check the manual.
Furthermore, you may want to see the traffic you’re receiving: in this case, use the command tcpdump seen during the course.
Init scripts
Init scripts are used to start processes at boot time (e.g. the web server nginx, SSH server and so on). Init scripts for system services are located in the /etc/init.d/ folder: if you want to start, stop or restart a service it is enough to use the command /etc/init.d/<service> <mode> where <service> is the name of the service on which you want to operate and <mode> is the operation type (e.g. start, stop or restart to perform the three operations listed above).
If you want to add or remove an init script, use the command rc-update with the add or del argument, followed by the service and the runlevel (typically default is used as runlevel). For instance:
root@diff:~# rc-update add nginx default
If you want to add custom init scripts, you may add them into /etc/local.d/.
For additional information about custom init scripts, you may refer to the related Gentoo documentation page.
Remote file copy
If you want to copy some files from your virtual machine to the computer you’re using in the lab or viceversa, scp is what you need. As for the standard cp command, first you have to specify the source and then the target: to denote a remote location you have to use the syntax <user>@<IP_or_domain_name>:<path>.
For instance, if you want to copy a file to a remote machine, you can proceed as follows:
MBPro:Desktop Mauro$ scp help.txt diff@192.168.69.22:/home/diff
diff@192.168.69.22's password:
help.txt 100% 207 0.2KB/s 00:00
Viceversa, if you want to copy a remote file in your working directory:
MBPro:Desktop Mauro$ scp diff@192.168.69.22:/home/diff/help.txt .
diff@192.168.69.22's password:
help.txt 100% 207 0.2KB/s 00:00
scp allows also to copy recursively entire directory trees (using the option -r) and so on: check the manual if you need more advanced options.
Command Line Editors
Copying a file to your computer each time you have to modify it and then moving it back to the virtual machine is not only boring: you also have to pay attention that the attributes of the copied file, such as user and group owner or permissions, are the same of the previous version. Using a command line editor is not a bad idea at all: one of the simplest editors available is nano. If you want to create or open a file, simply type nano <path_to_file> on the command line, while to perform commands like saving the file, quit from the program and so on, you need to know the relative combinations of keys: this page lists the ones used to perform the most common actions.
There are also more advanced command line text editors, such as vim: you’re free to use it if you want to make lavish happy, but remember that you have only 3 months to play the CTF 😀
Tcpserver
In the CTF we use tcpserver to allow the use of C-compiled programs by remote users without having to deal manually with sockets: when a connection is received on the port associated to a certain service, tcpserver runs the program with file descriptors 0 and 1 (that denote standard input and standard output) reading from and writing to the network. Suppose you have the following program:
#include
#include
int main(int argc, char *argv[]) {
char s[100];
printf("What's your name? ");
fflush(stdout);
scanf("%99s", s);
printf("Hi %s, how are you?\n", s);
return EXIT_SUCCESS;
}
You can bind the service to a port on a specific address typing tcpserver <options> <ip_address> <port> <absolute_path_to_executable>. For instance:
MBPro:~ Mauro$ tcpserver -H 0 12345 ~/name
Now you can use netcat, for example, to interact with the service:
MBPro:~ Mauro$ nc localhost 12345
What's your name? Mauro
Hi Mauro, how are you?
Actually, during the CTF you shouldn’t need to run the command tcpserver by yourself (unless, for some reason, tcpserver crashes): when you fix a vulnerability in a C program, it is enough to replace the executable of the service with the new one to use it.
Interaction with MySQL via command line
During the competition you may need to interact with the MySQL DBMS installed on your virtual machine for different reasons: for instance, to find out where are located the flags of services that rely on a database for data storage, in order to plan your attack.
You can open a simple SQL shell using the command mysql -u <user> -p, where user is the name of a user registered in the DBMS: you may also specify directly the password of the user after the option -p or insert it interactively.
Here are some useful commands: to see databases available on the system, you can use show databases.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| test |
+--------------------+
4 rows in set (0.00 sec)
If you want to see information about tables of a certain database or columns of a particular table, you can proceed as follows.
mysql> show tables from test;
+----------------+
| Tables_in_test |
+----------------+
| teams |
| users |
+----------------+
2 rows in set (0.00 sec)
mysql> show columns from test.users;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| id_user | int(10) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(30) | NO | | NULL | |
| surname | varchar(50) | NO | | NULL | |
| nick | varchar(20) | NO | UNI | NULL | |
| password | varchar(32) | NO | | NULL | |
| id_team | int(10) unsigned | NO | MUL | NULL | |
+----------+------------------+------+-----+---------+----------------+
6 rows in set (0.00 sec)
If you want to use a particular database, so that each time you don’t have to specify the DB name before the table on which you want to operate, give the command use <database>. Of course you can also execute standard SQL queries to retrieve, insert, update or delete data.
mysql> use test;
Database changed
mysql> select u.name, u.surname, t.name as 'team name' from users u, teams t where u.id_team=t.id_team;
+-------+-----------+----------------+
| name | surname | team name |
+-------+-----------+----------------+
| Mauro | Tempesta | c00kies@venice |
| Marco | Squarcina | c00kies@venice |
+-------+-----------+----------------+
2 rows in set (0.00 sec)
If you need to know more about this tool, on the web you will find a lot of information 🙂
HTTP requests with Python
When you find a flag, you have to submit it on a login protected site to earn points and rise in ranking: doing it manually for all the flags is really annoying, since you will have to collect as much flags as you can during the different rounds in which the competition is divided.
Manual submission is ok at the beginning of the competition, but remember that you are expected to write your own scripts to perform attacks automatically at each round against all your opponents. You can easily perform HTTP requests using a Python module that is installed on your virtual machines, Requests.
Consider a website that consists of the following pages:
- login.php: if nothing is received via POST, a form with fields username and password is shown, otherwise the script checks the correctness of authentication information. If credentials correspond to an existing user, the access to the page submit.php is granted.
- submit.php: if nothing is received via POST, a form with fields flag and service is shown, otherwise the script performs controls on the flag and, if there are no errors, it is marked as submitted. An error message is shown if a request comes from a non authenticated user.
Here is a simple script that performs authentication and submits a flag.
#!/usr/bin/python
import requests
session = requests.Session()
session.post("http:///login.php", data={"username": "nick", "password": "pwd"})
session.post("http:///submit.php", data={"flag": "FLG#djGfkng", "service": "pwnme"})
First of all a Session object is created: this kind of object allows you to execute multiple requests without having to deal manually with cookies that are used for session management. Then the script executes a POST to the login page, using the parameters required for authentication, and then submits the flag related to a certain service. Notice that the parameter data (a dictionary) is indispensable when you do a POST.
When you do a request (GET or POST), the method returns an object that represents the answer. You can print the content of the response (without HTTP headers, so the HTML code of the page) as follows:
response = session.get("http:///page.php")
print(response.text)
If you need more information, you can refer to the documentation on the website of the module, linked above.
Sockets with python
Communication over a network is achieved through sockets, which represent the communication endpoints used by programs to send and receive messages, similarly to file descriptors and other system resources.
We will focus on the IPv4 address family (AF_INET) and on stream sockets (SOCK_STREAM) as they are the most commonly used ones. The following example creates a stream IPv4 socket and use it to connect to www.google.com:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('www.google.com',80))
Bytes can be sent and received through send and recv primitive. However it is important to remember that these primitives work with the network buffer and it is not guaranteed that all the bytes are sent or received after one invocation. It is the programmer responsibility to invoke the functions many times, until all the bytes have been transmitted.
In order to experiment with sockets we run nc listening on port 10000:
$ nc -l -p 10000
Then we connect to it and we send string ‘ciao’ as follows:
>>> import socket
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.connect(('localhost',10000))
>>> s.send('ciao'.encode())
4
Notice that send returns the number of bytes read, 4 in this case. So we know that all the bytes haves been transmitted. On the shell we see that ‘ciao’ is correctly received by nc:
$ nc -l 10000
ciao
Since we are not guaranteed that all bytes are sent in one send we need to iterate until the whole string has been sent, as follows:
def mysend(s,msg):
totalsent = 0 # total number of bytes sent
while totalsent < len(msg): # until we send all the bytes
sent = s.send(msg[totalsent:].encode()) # sends the remaining bytes
if sent == 0:
raise RuntimeError("socket connection broken")
totalsent = totalsent + sent # updates the number of bytes sent
The above function counts how many bytes have been sent so far and goes on until the whole string has been sent. Notice the use of msg[totalsent:] to send what has not yet been sent.
Similarly, when we receive a message we need to iterate until all the bytes have been received. Sometimes, instead of reaching a specific number of bytes it is useful to read until a certain string is received (for example a newline). The following function checks both length and a specific string:
def myreceive(self,msglen,pattern=''):
'''receives up to msglen bytes and until pattern has been received
'''
msg = ''
while len(msg) < msglen and (pattern == '' or pattern not in msg):
chunk = self.sock.recv(msglen-len(msg)) # reads remaining bytes
if chunk == b'':
raise RuntimeError("socket connection broken")
msg = msg + chunk.decode() # append to already received bytes
return msg
For example if we invoke:
myreceive(s,100,'\n')
The function will loop until 100 bytes are read or a newline is found in the bytes read so far. We can experiment again with nc, writing an answer back to our program. We obtain the following:
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.connect(('localhost',10000))
>>> mysend(s,'ciao\n')
>>> print(myreceive(s,100,'\n'))
ciao ciao 123
The string 'ciao ciao 123' is what we have written on the terminal:
nc -l 10000
ciao (received)
ciao ciao 123 (written by the user)
The following code implements a Python class with basic functions to quickly interact with a remote application
import socket
class mysocket:
'''
slightly modified version of the one in
http://docs.python.org/2/howto/sockets.html
this runs under both Python2 and Python3
'''
def __init__(self, sock=None):
if sock is None:
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
else:
self.sock = sock
def connect(self, host, port):
self.sock.connect((host, port))
def mysend(self, msg):
totalsent = 0
while totalsent < len(msg):
sent = self.sock.send(msg[totalsent:].encode())
if sent == 0:
raise RuntimeError("socket connection broken")
totalsent = totalsent + sent
def myreceive(self,msglen,pattern=''):
'''receives up to msglen bytes and until pattern has been received
'''
msg = ''
while len(msg) < msglen and (pattern == '' or pattern not in msg):
chunk = self.sock.recv(msglen-len(msg))
if chunk == b'':
raise RuntimeError("socket connection broken")
msg = msg + chunk.decode()
return msg
Simple echo server
The following example shows a simple multi-threaded echo server. In particular:
- the primitive
bindbinds the server socket to a specific address; - the primitive
listenlistens for connections; - new connections are accepted through the primitive
acceptwhich returns a new socket for each new connection. This allows for independently dealing with multiple connections.
Notice that each connection is handled by a different thread so to allow many connections at a time.
# Simple echo server multi threaded
import socket
from threading import Thread
import mysocket
# this is run in a new thread
def connessione(socket):
s = mysocket.mysocket(socket)
print ('New connection from host {} port {}'.format(*socket.getpeername()))
# echo server
while True:
r = s.myreceive(1)
if r == '':
break
s.mysend(r)
print ('Connection closed')
def main():
# create an INET, STREAMing socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# bind the socket to a public host and a port
serversocket.bind(('localhost', 8000))
# become a server socket
serversocket.listen(5)
while 1:
# accept connections
(clientsocket, address) = serversocket.accept()
# we start a new thread so that we can handle multiple connections!
t = Thread(target=connessione, args=(clientsocket,))
t.start()
if __name__ == "__main__":
main()
We can run the server and connect to it using nc from multiple terminals. Everything we write will be sent back. Connections will be dealt with independently. For example:
$ nc localhost 8000
test (sent)
test (received)
test2 (sent)
test2 (received)
Exercises
- Run the simple echo server and connect to it from multiple
ncclients. Notice that this is possible because of multithreading - Remove multi-threading and observe that now the server can only handle one connection at a time
- Write a program that connects to the echo server, sends a string and reads the answer
- Modify the echo server so that it changes some words before echoing them back (note that this requires to change the way the server reads which, in the example, is 1 byte at the time)