« Archives in January, 2011

Anti Anti Debugging Tricks

Housekeeping: Well… It’s been a while since I have posted. I had been meaning to post some things but Google turned off their FTP service in Blogger making their service really useless to me and so this blog languished while I finally got around to sorting things out and moving it to a proper blog software. I hope to be providing more updates now, but I won’t fool myself too much.

On with the show….

I recently came across a binary application on a Linux server that had been coded to prevent people from snooping into what the application actually did. If you ran the application using strace or gdb the application would detect it and stop running. It would throw an error similar to:

“Debugging detected… goodbye!”

and the application would simply exit. Now I understand a programmer’s desire to protect their code, however if you are going to be running an application on my server I should at least know what it is you are actually doing. Apparently this is known as “anti debugging” and is designed to prevent reverse engineering of an application. Not being one to turn down a challenge…. I accepted. Below I will outline some very simple processes that can be used to circumvent some of the more basic checks.

There are apparently a few different methodologies involved in anti debugging, and it seems that our application used a couple of checks. I will show you some of what the application was doing and the very simple workarounds to get around them.

When you attach a debugging tool such as strace or gdb to the command it would throw off the process id of the parent application. Apparently the application was checking that by using a ptrace call to attach to the parent process id (getppid). Here is how I worked around this issue.

We override the getppid system call with our own. Create a very simple C application that will define our new getppid() calls.

#include <stdio.h>

int getppid() { return getsid( getpid() ); }

What this does is return the session id for current process’s id anytime getppid() is called. So now we need to compile this into a shared library that we can load before we execute the application we want to debug.

gcc -shared -fPIC -o fakegetppid fakegetpppid.c

This compiles the C application into a shared library that we can then load. Here is how we tell something like strace to use this instead of the normal getppid system call.

strace -fxi -E LD_PRELOAD=./fakegetppid /path/to/the/main/application

If there are no other anti-debugging tricks in place, the application will execute as normal.

What if you have an anti-debugging trick that isn’t so simple to thwart? What if the application makes multiple calls to a system call that you need to catch a specific instance of? Well there is a way to accomplish that as well.

#define _GNU_SOURCE 1

#include <stdio.h>
#include <dlfcn.h>

static int(*next_ptrace)(int a, int b, int c, int d ) = NULL;

long ptrace( int a, int b, int c, int d )
{
    if( next_ptrace == NULL ) {
        next_ptrace = dlsym( RTLD_NEXT, "ptrace" );
    }

    if( a == 16 ) { /* PTRACE_ATTACH */
        fprintf( stderr, "PTRACE_ATTACH called with pid %i\n", b);
    }

    return next_ptrace( a, b, c, d );

}

This will allow you to specifically capture PTRACE_ATTACH calls and do whatever you want with the calls, sending all other calls to the original system ptrace call. That’s pretty darn powerful and works exactly the same way as the code above.

I hope this information is useful to some people out there. I know it is a little complex, but it just goes to show that it is possible to debug an application, even if it doesn’t appear to be at first.

Good luck and happy hunting.