Tips for the CTF

The aim of this page, which may be updated during the semester, is to give you some hints for the CTF. Remember that each team has to manage its own virtual machine, running some vulnerable services on a Gentoo Linux operating system, hence you may need to know some useful commands to list or kill processes currently in execution, find out active connections on your machine, connect to a MySQL database via command line and so on.

Furthermore, during the CTF you should develop some scripts to avoid to perform manually the attacks and the submission of the flags during each round, so we will show you how to perform easily HTTP requests using Python.

Clearly this cannot be a comprehensive guide: if you need more information, please refer to the manual page of the related command or ask Google 😀
If you think that something really important is missing or you haven’t understood something, feel free to post a comment!

Change your password

The first thing you have to do when the competition starts is to change the root password of your virtual machine: since you will connect directly as root with ssh, it is enough to use the command passwd.

root@diff:~# passwd
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully

Switch user

In your machine there will be a different user for each service: since it is a good habit to use root only when strictly necessary, you may consider the possibility of switching to the user that owns the service on which you want to work. This can be done with the command su - ; if you want to go back to the previous user, you can use the combination CTRL + D or the command exit.

root@diff:~# su - pin
pin@diff:~$ id
uid=1001(pin) gid=1001(pin) groups=1001(pin)
pin@diff:~$ exit
logout
root@diff:~# 

The manual

If you need to know more information about a command, you must simply type man <command_name> on the command line. You can search for a word typing /<word> once the manual page is open and, if you press n, the next occurrence will be shown. You can move inside the page using arrow keys. When you have finished, type q to exit.

You can also get the documentation of many functions of the standard C library using man 3 <function_name>.

Do you want to know more about man? Read the manual of the manual 🙂

Files search

find is a very powerful tool for searching files inside a given path with some particular properties: for instance you can ask to list files that belong to a given user, search for files on which you have particular permissions and so on. You can also combine different properties using logical operators.

For instance, this command allows you to find regular files (not directories) owned by the specified user that are readable by everybody:

diff@diff:~$ find /home/pin/ -user pin -perm /a+r -type f
/home/pin/challenge/pin.py
/home/pin/challenge/server.py
/home/pin/challenge/pin.pyc

Instead, if you want to search, only in the working directory, for files whose names start with .b or with permissions exactly 755, we can use:

diff@diff:~$ find . -maxdepth 1 -name '\.b*' -o -perm 755
.
./.bash_history
./.bash_logout
./.bashrc

Refer to the manual if you want to know more about this utility.

Process management

A common way to obtain complete information about the processes currently in execution on the system consists on using ps with options aux. Here is an example of (part of) the result produced by the command:

diff@diff:~$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.3   3504  1764 ?        Ss   Feb11   0:02 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Feb11   0:01 [kthreadd]
diff       238  0.0  1.6  14808  8444 ?        S    Feb11   0:01 /usr/bin/python /home/diff/challenge/server.py
diff     17417  0.1  0.2   4936  1232 pts/2    T    18:59   0:00 nano
diff     17418  0.0  0.2   5204  1192 pts/2    R+   18:59   0:00 ps aux
diff     32098  0.0  1.2  12400  6256 pts/4    S+   Apr17   7:44 python -m SimpleHTTPServer

The fields to which you may be interested are probably USER, PID and COMMAND that respectively denote, for each process, the user that has started it, its process identifier and the command given to start it.

You can kill a process using the command kill and specifying the PID: for instance, if we want to kill nano:

diff@diff:~$ kill -9 17417
diff@diff:~$ ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.3   3504  1764 ?        Ss   Feb11   0:02 /sbin/init
root         2  0.0  0.0      0     0 ?        S    Feb11   0:01 [kthreadd]
diff       238  0.0  1.6  14808  8444 ?        S    Feb11   0:01 /usr/bin/python /home/diff/challenge/server.py
diff     17425  0.0  0.2   5204  1192 pts/2    R+   19:00   0:00 ps aux
diff     32098  0.0  1.2  12400  6256 pts/4    S+   Apr17   7:44 python -m SimpleHTTPServer

If you want to kill processes that are not owned by your current user, you have to switch to the one that owns the process (or to root).

You can even specify to kill processes with a (partially) given name or that belong to some specified users and so on using pkill: please refer to the manual for more information.

Network connections

Information about active network connections can be seen using netstat as follows:

diff@diff:~$ netstat -natup
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:8000            0.0.0.0:*               LISTEN      32098/python    
tcp        0      0 0.0.0.0:42765           0.0.0.0:*               LISTEN      238/python      
tcp        0      0 0.0.0.0:42766           0.0.0.0:*               LISTEN      -               
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -               
tcp        0    672 192.168.69.22:22        192.168.69.159:51987    ESTABLISHED -               
tcp6       0      0 :::22                   :::*                    LISTEN      -  

Column Proto reports the transport protocol in use, Local Address and Foreign Address denote, respectively, addresses of the parts involved in the connection, State is the state of the connection (except for connectionless protocols) and PID/Program name shows the PID and the name of the program that has created the connection. The last column shows information only of processes owned by the user that gives the command: if you want to see them all, you have to run the command as root.

For instance, the first row says that there is a Python program listening on the port 8000, while the fifth says that there is a connection on the interface with IP address 192.168.69.22 on the port 22 with a machine with address 192.168.69.159 on the port 51987.

As we have seen during the course, you can quickly open a connection with an entity using nc (netcat), specifying the IP address (or the domain name) and the port. An example follows, referred to a service of the previous year:

diff@diff:~$ nc 192.168.69.22 42765
Insert your 5 digits user ID (trailing zeros allowed) for the generation of the new password: 12345
New password: nz41HgDXQ7
Data to be stored in the key: 750f286135dc1dc745d417a2482a225b219b96c2d6401b2b

Apart from this, you can also use nc to listen for a connection and so on: if you want to know more, check the manual.

Furthermore, you may want to see the traffic you’re receiving: in this case, use the command tcpdump seen during the course.

Init scripts

Init scripts are used to start processes at boot time (e.g. the web server nginx, SSH server and so on). Init scripts for system services are located in the /etc/init.d/ folder: if you want to start, stop or restart a service it is enough to use the command /etc/init.d/<service> <mode> where <service> is the name of the service on which you want to operate and <mode> is the operation type (e.g. start, stop or restart to perform the three operations listed above).

If you want to add or remove an init script, use the command rc-update with the add or del argument, followed by the service and the runlevel (typically default is used as runlevel). For instance:

root@diff:~# rc-update add nginx default

If you want to add custom init scripts, you may add them into /etc/local.d/.

For additional information about custom init scripts, you may refer to the related Gentoo documentation page.

Remote file copy

If you want to copy some files from your virtual machine to the computer you’re using in the lab or viceversa, scp is what you need. As for the standard cp command, first you have to specify the source and then the target: to denote a remote location you have to use the syntax <user>@<IP_or_domain_name>:<path>.
For instance, if you want to copy a file to a remote machine, you can proceed as follows:

MBPro:Desktop Mauro$ scp help.txt diff@192.168.69.22:/home/diff
diff@192.168.69.22's password: 
help.txt                                                                                        100%  207     0.2KB/s   00:00  

Viceversa, if you want to copy a remote file in your working directory:

MBPro:Desktop Mauro$ scp diff@192.168.69.22:/home/diff/help.txt .
diff@192.168.69.22's password: 
help.txt                                                                                        100%  207     0.2KB/s   00:00

scp allows also to copy recursively entire directory trees (using the option -r) and so on: check the manual if you need more advanced options.

Command Line Editors

Copying a file to your computer each time you have to modify it and then moving it back to the virtual machine is not only boring: you also have to pay attention that the attributes of the copied file, such as user and group owner or permissions, are the same of the previous version. Using a command line editor is not a bad idea at all: one of the simplest editors available is nano. If you want to create or open a file, simply type nano <path_to_file> on the command line, while to perform commands like saving the file, quit from the program and so on, you need to know the relative combinations of keys: this page lists the ones used to perform the most common actions.

There are also more advanced command line text editors, such as vim: you’re free to use it if you want to make lavish happy, but remember that you have only 3 months to play the CTF 😀

Tcpserver

In the CTF we use tcpserver to allow the use of C-compiled programs by remote users without having to deal manually with sockets: when a connection is received on the port associated to a certain service, tcpserver runs the program with file descriptors 0 and 1 (that denote standard input and standard output) reading from and writing to the network. Suppose you have the following program:

#include 
#include 

int main(int argc, char *argv[]) {
	char s[100];

	printf("What's your name? ");
	fflush(stdout);
	scanf("%99s", s);
	printf("Hi %s, how are you?\n", s);

	return EXIT_SUCCESS;
}

You can bind the service to a port on a specific address typing tcpserver <options> <ip_address> <port> <absolute_path_to_executable>. For instance:

MBPro:~ Mauro$ tcpserver -H 0 12345 ~/name

Now you can use netcat, for example, to interact with the service:

MBPro:~ Mauro$ nc localhost 12345
What's your name? Mauro
Hi Mauro, how are you?

Actually, during the CTF you shouldn’t need to run the command tcpserver by yourself (unless, for some reason, tcpserver crashes): when you fix a vulnerability in a C program, it is enough to replace the executable of the service with the new one to use it.

Interaction with MySQL via command line

During the competition you may need to interact with the MySQL DBMS installed on your virtual machine for different reasons: for instance, to find out where are located the flags of services that rely on a database for data storage, in order to plan your attack.

You can open a simple SQL shell using the command mysql -u <user> -p, where user is the name of a user registered in the DBMS: you may also specify directly the password of the user after the option -p or insert it interactively.

Here are some useful commands: to see databases available on the system, you can use show databases.

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| test               |
+--------------------+
4 rows in set (0.00 sec)

If you want to see information about tables of a certain database or columns of a particular table, you can proceed as follows.

mysql> show tables from test;
+----------------+
| Tables_in_test |
+----------------+
| teams          |
| users          |
+----------------+
2 rows in set (0.00 sec)

mysql> show columns from test.users;
+----------+------------------+------+-----+---------+----------------+
| Field    | Type             | Null | Key | Default | Extra          |
+----------+------------------+------+-----+---------+----------------+
| id_user  | int(10) unsigned | NO   | PRI | NULL    | auto_increment |
| name     | varchar(30)      | NO   |     | NULL    |                |
| surname  | varchar(50)      | NO   |     | NULL    |                |
| nick     | varchar(20)      | NO   | UNI | NULL    |                |
| password | varchar(32)      | NO   |     | NULL    |                |
| id_team  | int(10) unsigned | NO   | MUL | NULL    |                |
+----------+------------------+------+-----+---------+----------------+
6 rows in set (0.00 sec)

If you want to use a particular database, so that each time you don’t have to specify the DB name before the table on which you want to operate, give the command use <database>. Of course you can also execute standard SQL queries to retrieve, insert, update or delete data.

mysql> use test;
Database changed
mysql> select u.name, u.surname, t.name as 'team name' from users u, teams t where u.id_team=t.id_team;
+-------+-----------+----------------+
| name  | surname   | team name      |
+-------+-----------+----------------+
| Mauro | Tempesta  | c00kies@venice |
| Marco | Squarcina | c00kies@venice |
+-------+-----------+----------------+
2 rows in set (0.00 sec)

If you need to know more about this tool, on the web you will find a lot of information 🙂

HTTP requests with Python

When you find a flag, you have to submit it on a login protected site to earn points and rise in ranking: doing it manually for all the flags is really annoying, since you will have to collect as much flags as you can during the different rounds in which the competition is divided.

Manual submission is ok at the beginning of the competition, but remember that you are expected to write your own scripts to perform attacks automatically at each round against all your opponents. You can easily perform HTTP requests using a Python module that is installed on your virtual machines, Requests.

Consider a website that consists of the following pages:

  • login.php: if nothing is received via POST, a form with fields username and password is shown, otherwise the script checks the correctness of authentication information. If credentials correspond to an existing user, the access to the page submit.php is granted.
  • submit.php: if nothing is received via POST, a form with fields flag and service is shown, otherwise the script performs controls on the flag and, if there are no errors, it is marked as submitted. An error message is shown if a request comes from a non authenticated user.

Here is a simple script that performs authentication and submits a flag.

#!/usr/bin/python

import requests

session = requests.Session()
session.post("http://
/login.php", data={"username": "nick", "password": "pwd"}) session.post("http://
/submit.php", data={"flag": "FLG#djGfkng", "service": "pwnme"})

First of all a Session object is created: this kind of object allows you to execute multiple requests without having to deal manually with cookies that are used for session management. Then the script executes a POST to the login page, using the parameters required for authentication, and then submits the flag related to a certain service. Notice that the parameter data (a dictionary) is indispensable when you do a POST.

When you do a request (GET or POST), the method returns an object that represents the answer. You can print the content of the response (without HTTP headers, so the HTML code of the page) as follows:

response = session.get("http://
/page.php") print(response.text)

If you need more information, you can refer to the documentation on the website of the module, linked above.

Sockets with python

Communication over a network is achieved through sockets, which represent the communication endpoints used by programs to send and receive messages, similarly to file descriptors and other system resources.

We will focus on the IPv4 address family (AF_INET) and on stream sockets (SOCK_STREAM) as they are the most commonly used ones. The following example creates a stream IPv4 socket and use it to connect to www.google.com:

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('www.google.com',80))

Bytes can be sent and received through send and recv primitive. However it is important to remember that these primitives work with the network buffer and it is not guaranteed that all the bytes are sent or received after one invocation. It is the programmer responsibility to invoke the functions many times, until all the bytes have been transmitted.

In order to experiment with sockets we run nc listening on port 10000:

$ nc -l -p 10000

Then we connect to it and we send string ‘ciao’ as follows:

>>> import socket
>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.connect(('localhost',10000))
>>> s.send('ciao'.encode())
4

Notice that send returns the number of bytes read, 4 in this case. So we know that all the bytes haves been transmitted. On the shell we see that ‘ciao’ is correctly received by nc:

$ nc -l 10000
ciao

Since we are not guaranteed that all bytes are sent in one send we need to iterate until the whole string has been sent, as follows:

def mysend(s,msg):
	totalsent = 0 # total number of bytes sent
	while totalsent < len(msg): # until we send all the bytes
		sent = s.send(msg[totalsent:].encode()) # sends the remaining bytes
		if sent == 0: 
			raise RuntimeError("socket connection broken")
		totalsent = totalsent + sent # updates the number of bytes sent

The above function counts how many bytes have been sent so far and goes on until the whole string has been sent. Notice the use of msg[totalsent:] to send what has not yet been sent.

Similarly, when we receive a message we need to iterate until all the bytes have been received. Sometimes, instead of reaching a specific number of bytes it is useful to read until a certain string is received (for example a newline). The following function checks both length and a specific string:

def myreceive(self,msglen,pattern=''):
    '''receives up to msglen bytes and until pattern has been received
    '''
    msg = ''
    while len(msg) < msglen and (pattern == '' or pattern not in msg):
        chunk = self.sock.recv(msglen-len(msg)) # reads remaining bytes
        if chunk == b'':
            raise RuntimeError("socket connection broken")
        msg = msg + chunk.decode() # append to already received bytes
    return msg

For example if we invoke:

myreceive(s,100,'\n')

The function will loop until 100 bytes are read or a newline is found in the bytes read so far. We can experiment again with nc, writing an answer back to our program. We obtain the following:

>>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>>> s.connect(('localhost',10000))
>>> mysend(s,'ciao\n')
>>> print(myreceive(s,100,'\n'))
ciao ciao 123

The string 'ciao ciao 123' is what we have written on the terminal:

nc -l 10000
ciao            (received)
ciao ciao 123   (written by the user)

The following code implements a Python class with basic functions to quickly interact with a remote application

import socket

class mysocket:
    '''
	    slightly modified version of the one in 
	    http://docs.python.org/2/howto/sockets.html
	    this runs under both Python2 and Python3
    '''

    def __init__(self, sock=None):
        if sock is None:
            self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        else:
            self.sock = sock
 
    def connect(self, host, port):
        self.sock.connect((host, port))
 
    def mysend(self, msg):
        totalsent = 0
        while totalsent < len(msg):
            sent = self.sock.send(msg[totalsent:].encode())
            if sent == 0:
                raise RuntimeError("socket connection broken")
            totalsent = totalsent + sent
 
    def myreceive(self,msglen,pattern=''):
        '''receives up to msglen bytes and until pattern has been received
        '''
        msg = ''
        while len(msg) < msglen and (pattern == '' or pattern not in msg):
            chunk = self.sock.recv(msglen-len(msg))
            if chunk == b'':
                raise RuntimeError("socket connection broken")
            msg = msg + chunk.decode()
        return msg

Simple echo server

The following example shows a simple multi-threaded echo server. In particular:

  • the primitive bind binds the server socket to a specific address;
  • the primitive listen listens for connections;
  • new connections are accepted through the primitive accept which returns a new socket for each new connection. This allows for independently dealing with multiple connections.

Notice that each connection is handled by a different thread so to allow many connections at a time.

# Simple echo server multi threaded
import socket
from threading import Thread
import mysocket

# this is run in a new thread
def connessione(socket):
	s = mysocket.mysocket(socket)
 
	print ('New connection from host {} port {}'.format(*socket.getpeername()))
	# echo server
	while True:
		r = s.myreceive(1)
		if r == '':
			break
		s.mysend(r)
	print ('Connection closed')
 
def main():
    # create an INET, STREAMing socket
    serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    # bind the socket to a public host and a port
    serversocket.bind(('localhost', 8000))
    # become a server socket
    serversocket.listen(5)
 
    while 1:
        # accept connections
        (clientsocket, address) = serversocket.accept()
        # we start a new thread so that we can handle multiple connections!
        t = Thread(target=connessione, args=(clientsocket,))
        t.start()
 
if __name__ == "__main__":
    main()

We can run the server and connect to it using nc from multiple terminals. Everything we write will be sent back. Connections will be dealt with independently. For example:

$ nc localhost 8000
test   (sent)
test   (received)
test2  (sent)
test2  (received)

Exercises

  1. Run the simple echo server and connect to it from multiple nc clients. Notice that this is possible because of multithreading
  2. Remove multi-threading and observe that now the server can only handle one connection at a time
  3. Write a program that connects to the echo server, sends a string and reads the answer
  4. Modify the echo server so that it changes some words before echoing them back (note that this requires to change the way the server reads which, in the example, is 1 byte at the time)