Part 1: Linux Kernel Debug and Exploitation

Introduction

One of Linus TORVALDS famous quote is :

The Linux philosophy is "Laugh in the face of danger". Oops. Wrong One. "Do it yourself". Yes, that's it.

In order to dig into the Linux Core Kernel, we need to build a complete and bootable Linux environment “our-self”, for that buildroot toolchain cross-compilation allows compiling Linux kernel image for multiple target platforms.

Debugging Linux kernel is different from debugging normal executable : normal applications (i.e process) always runs in security ring 3 ( user-mode layer), when the process invoke access to hardware resources, the request is sent to the kernel (security ring 0) via system calls and the kernel manage to do the task with low-level functions. To sum up, security rings exists to restrict privileges, and the kernel-mode plays the role of middleware between the hardware and the user-mode.

Usually when we use gdb For user-mode : gdb allow inspecting what is going inside an executable on run-time execution. For kernel-mode : gdb allow debugging the low-level system call functions and module loaded.

Set up Kernel Image

Typically, when we debug or exploit a kernel module we do that either on Virtual Machine or on an emulator like QEMU. (A simple bad handling can crash the host kernel and we don’t want to deal with PANIC errors). QEMU can do perfectly the job, so :

0
1
2
$ apt search qemu
qemu-system QEMU full system emulation binaries
# apt install qemu-system

Buildroot is often used for building embedded systems that target processors other than the regular x86, like ARM and MIPS processors. In our use-case, we will focus only on the x86 architecture. All build process has been conducted on :

Linux distribution release: Ubuntu 16.04 LTS
Kernel version: 4.15.0–55

First, download and extract the source files from the downloaded tar archive :

0
1
2
$ curl -JLO https://buildroot.org/downloads/buildroot-2016.08-rc2.tar.bz2
$ tar -xvf buildroot-2016.08-rc2.tar.bz2
$ cd buildroot-2016.08-rc2

1 ] Building Part

You’re free to make your choice : Build for 64-bit :

0
1
$ make qemu_x86_64_defconfig
$ make menuconfig

In Target options > Target architecture > x86_64

In Build options > select build packages with debugging symbols
In Build options > build packages with debugging symbols > gcc debug level and select level 3
In Build options > build packages with debugging symbols > strip command for binaries on target and select none
In Build options > build packages with debugging symbols > gcc optimization level and select optimize for debugging

For building, run the command line:

0
$ make

Build for 32-bit :

0
1
$ make qemu_x86_defconfig
$ make menuconfig

In Target options > Target architecture > i386

And the configuration of Build options is the same as 64-bit

2 ] Kernel Configuration Part

Run the command below for linux configuration :

0
$ make linux-menuconfig

In Kernel hacking > select Kernel debugging
In Kernel hacking > Compile-time checks and compiler options > select Compile kernel with debug info and select Provide GDB scripts for kernel debugging

Or, we can just run the command line:

0
$ ./output/build/linux-4.7.1/scripts/config -e DEBUG_INFO -e GDB_SCRIPTS

For building, run the command line:

0
$ make

3 ] Booting with QEMU

If all goes well, the compiled Linux kernel image should boot with QEMU. For booting the image with QEMU run:

0
1
2
3
4
5
6
7
// For 32 bit
$ qemu-system-i386 -kernel output/images/bzImage \
                   -hda output/images/rootfs.ext2 \
                   -append "root=/dev/sda rw" -s
// For 64 bit 
$ qemu-system-x86_64 -kernel output/images/bzImage \
                   -hda output/images/rootfs.ext2 \
                   -append "root=/dev/sda rw" -s

Remember: the argument -s refers to a shorthand for -gdb tcp::1234. Congratulations, the first step done!! 🎉 🎉

Basic Kernel Debugging

Now that we can boot the Linux kernel image in QEMU, let’s start gdb and attach to gdb stub of QEMU gdbsever running on the kernel image on default port :1234 An additional step left before begin debugging, by default Ubuntu restrict auto-loading of GDB scripts: remember in “Building Kernel Step”, we have configured the kernel to provide GDB scripts for kernel debugging, those scripts are useful helper when debugging kernel modules.

Those scripts are located in ./output/build/linux-<version>/scripts/gdb/ We have to allow loading scripts by adding the line below into the config file of gdb located in ~/.gdbinit

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
$ echo "set auto-load safe-path /path/to/build/script" >> ~/.gdbinit
$ cd output/build/linux-4.7
// Run gdb and load vmlinux 
$ gdb 
gef➤ file vmlinux
// Load kernel modules symbols  
gef➤ lx-symbols
loading vmlinux
// Attach to the target guest on port '1234'
gef➤ target remote :1234
gef➤ c
(continue)...

From now on, the kernel image is paused, we can resume the qemu kernel with gdb command c, and pause it again with the shortcut Ctrl+c .

In order to understand the basics core concepts of the Linux kernel, first and foremost we should cover the concept of process management.

Process Management

The two of the most important structures in the kernel are struct thread_info and struct task_struct . From the kernel point of view about process descriptors:

the user-space process is a task and the kernel allocates one task_struct object on memory for each task
With user-space and kernel-space threads, the kernel allocates one task_struct object for every thread running.

The structure objects are allocated with the memory management mechanism, either we can use SLAB or SLUB allocator. SLAB/SLUB use the model of object caching to reduce memory fragmentation caused by allocations and deallocations operations.

By default, the memory management used when building kernel is SLUB: the next generation of SLAB that improves performance.

linux-menuconfig

To print the description of the structure thread_info:

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
// In gdb
gef➤ ptype struct thread_info
type = struct thread_info {
    struct task_struct *task;
    __u32 flags;
    __u32 status;
    __u32 cpu;
    mm_segment_t addr_limit;
    unsigned int sig_on_uaccess_error : 1;
    unsigned int uaccess_err : 1;
}
gef➤

The thread_info struct hold 2 important fields:

task’s stack pointer
addr_limit used to separate between user and kernel spaces

The main purpose of the global variable addr_limit is to allow unprivileged functions to read or write from or to kernel-space memory.

Let’s take an example:

The executable ifconfig use ioctl to set and get the configuration of the network device.

The kernel-space use ./net/ipv4/ipconfig.c module to configure the network device, then use the function devinet_ioctl() to create an info request struct ifreq ifr structure and copies data from user to kernel space with copy_from_user()

./net/ipv4/ipconfig.c

./net/ipv4/devinet.c

To transfer data from user to kernel space the function copy_from_user() must be used in order to access the user-space pointer (in this case void __user *arg ; the kernel uses __user to identify pointers of user-space).

Before calling copy_from_user() we should set the value of the variable addr_limit. The sequence code temporarily raises addr_limit so that any function with copy_from_user(), which is normally restricted reading data into user-space memory, can read write into a kernel-space pipe buffer:

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
// Put the current addr_limit into "old_FS"
old_FS = get_fs(); 

// Set the addr_limit into KERNEL_DS==0xffffffffffffffff
set_fs(get_ds());  

// Call the function that use copy_from_user()
...

// Restore back the addr_limit into "old_FS"
set_fs(old_FS);   

./arch/x86/include/asm/uaccess.h

GDB in Action

Get the value of addr_limit using gdb, When we tape ifconfig for example as a command line, the shell fork() , execve() and wait() the process. execve() uses the system call sys_execve() as an entry point which does reference to the function do_execve() (execve() -> sys_execve() -> do_execve()).

The gdb command info functions will display all the loaded symbols of the kernel, for a matching regex like execve : info functions execve

Note

Kernel modules don’t execute sequentially as applications do, most actions performed by the kernel are related to a specific task. Kernel code can know the current task driving it by accessing the macro current, a pointer to the struct task_struct. The current pointer refers to the user process (task) currently executing. During the execution of a system call, such as open or read, the current process is the one that invoked the call. Kernel code can use process-specific information by using current, if it needs to do so.

The thread_info structure lives at the bottom of the kernel stack, it can be acquired by calculating the address of the RSP register:

0
1
2
3
4
static inline struct thread_info *current_thread_info(void)
{
        register unsigned long sp asm ("sp");
        return (struct thread_info *)(sp & ~(THREAD_SIZE - 1));
}

The kernel stack … is fixed in size. The exact size of the kernel’s stack varies by architecture. On x86, the stack size is configurable at compile time and can be either 4KB or 8KB. Historically, the kernel stack is two pages, which generally means that it is 8KB on 32-bit architectures and 16KB on 64-bit architectures - this size is fixed and absolute. Each process receives its own stack…

In order to feel reassured, compile and execute the following C program:

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
/*
** compile => gcc stacky.c -o stacky
*/
#include <stdio.h>
#include <linux/const.h>
#include <linux/types.h>
#define PAGE_SHIFT              12
#define PAGE_SIZE               (_AC(1,UL) << PAGE_SHIFT)
#define THREAD_SIZE_ORDER       2
#define THREAD_SIZE  (PAGE_SIZE << THREAD_SIZE_ORDER)
#define CURRENT_MASK (~(THREAD_SIZE - 1))
int main(){
        printf("page size : 0x%lx\n",PAGE_SIZE);
        printf("page size : %ld\n",PAGE_SIZE);
        printf("thread size : 0x%lx\n",THREAD_SIZE);
        printf("thread size : %ld\n",THREAD_SIZE);
        printf("CURRENT_MASK : 0x%lx\n",CURRENT_MASK);
}

The kernel stack output:

0
1
2
3
4
page size : 0x1000
page size : 4096
thread size : 0x4000
thread size : 16384
CURRENT_MASK : 0xffffffffffffc000

After confirming the thread size needed for accessing the thread_info structure, with the gdb commands bellow we can access the structure fields :

Or, just using the loaded functions of the gdb scripts helper, we can recover the same result:

The field [addr_limit] between the Kernel and User Space

- - - -

`task_struct` Structure

task_struct contains all the information about the current task,

 0
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
gef➤  ptype struct task_struct
type = struct task_struct {
    volatile long state;
    void *stack; //thread_info pointer
    atomic_t usage;
    unsigned int flags;
    unsigned int ptrace;
    ...
    pid_t pid;
    pid_t tgid;
    struct task_struct *real_parent; //real parent process
    struct list_head children;  // list of task children
    ...   
    u64 start_time;
    u64 real_start_time;
    ...
    const struct cred *real_cred;
    const struct cred *cred; // process credentials(uid,gid...)
    char comm[16]; //executable name excluding path
    ...
    struct fs_struct *fs;
    struct files_struct *files;
    ...
    struct thread_struct thread;
}

The task pointer in thread_info is actually the pointer to the task_struct structure, and the stack pointer in task_struct refers to the thread_info structure. Thus, if we have one of the 2 pointers (cf. task_struct or thread_info ), we can access the other.

For example, the structure cred can be used as a part of security check performed by the kernel upon Linux objects (tasks, files, …). The task credentials holds uid=0 gid=O as the user is root :

`thread_info` Clean up

For kernel version 4.8 or later, the structure thread_info has been cleaned up, most of thread_info fields were moved into other kernel structures: addr_limit has been moved into thread_struct

Part 1: Linux Kernel Debug and Exploitation

Dive deep into linux kernel source code

Table Of Contents

Introduction

Set up Kernel Image

1 ] Building Part

2 ] Kernel Configuration Part

3 ] Booting with QEMU

Basic Kernel Debugging

Process Management

Let’s take an example:

GDB in Action

Note

The field [addr_limit] between the Kernel and User Space

`task_struct` Structure

`thread_info` Clean up

Table Of Contents

Introduction

Set up Kernel Image

1 ] Building Part

2 ] Kernel Configuration Part

3 ] Booting with QEMU

Basic Kernel Debugging

Process Management

Let’s take an example:

GDB in Action

Note

The field [addr_limit] between the Kernel and User Space

task_struct Structure

thread_info Clean up

`task_struct` Structure

`thread_info` Clean up