In between midnight summer nights and dark winter days

Running iTunes in a debugger (gdb) Thu, Jul 28 2011 11:04

iTunes is one of a few Apple-provided applications that prevent you from attaching a debugger. This can be problematic if you are, say, developing an iTunes visualizer. iTunes accomplishes this by calling ptrace(PT_DENY_ATTACH, 0, 0, 0), which prevents future attempts at tracing the process. Debuggers like gdb use ptrace to control the process they debug. Luckily, there's a way to intercept the ptrace-call before it completes, simply by starting iTunes in the debugger. Let's see what happens when we run iTunes from gdb in Terminal, without doing anything:

[#] cd /Applications/
[#] gdb ./iTunes
gdb shows some license info
(gdb) run
Starting program: /Applications/ 
Reading symbols for shared libraries ... done
Program exited with code 055.

Not quite what we want. Let's see what happens when we put a breakpoint on ptrace before starting iTunes:

(gdb) br ptrace
Breakpoint 1 at 0x7fff90592d14
(gdb) run
Starting program: /Applications/ 
Reading symbols for shared libraries . done

Breakpoint 1, 0x00007fff90592d14 in ptrace ()

We're stopped at the entry to the ptrace() function. Now, at this point we have two options: We can force the function to return, setting its return value to 0 to indicate no error, or we can modify the arguments to the ptrace call. Either way works, but this post will demonstrate the latter approach.

We know that iTunes calls ptrace with PT_DENY_ATTACH as its argument. Looking at the ptrace header file, we discover that PT_DENY_ATTACH corresponds to the value 31:

From /usr/include/sys/ptrace.h:
#define PT_DENY_ATTACH 31

Now, let's look at the contents of the registers:

(gdb) info registers
rax            0x24a0   9376
rbx            0x7fff5fc35120 140734800023840
rcx            0x0      0
rdx            0x0      0
rsi            0x24a0   9376
rdi          0x1f   31
Rest of register contents snipped

The register rdi looks promising. Let's try changing it to the value matching PT_DETACH:

(gdb) set $rdi=11
(gdb) info registers
rax            0x24a0   9376
rbx            0x7fff5fc35120 140734800023840
rcx            0x0      0
rdx            0x0      0
rsi            0x24a0   9376
rdi          0xb    11
(gdb) continue
Reading symbols for shared libraries . done
Symbols keep loading, and iTunes launches. And we're still attached!

So that seemed to work brilliantly. At this point, you can use the debugger as normal, pausing iTunes when you need to, getting backtraces, and catch errors in your visualizer. The only caveat is that detaching iTunes using gdb will probably hang, since we replaced PT_DENY_ATTACH with PT_DETACH earlier. This can be worked around by using the "return immediately" approach I alluded to earlier; doing this will be left as an exercise for the reader. To make gdb exit, you will probably have to kill it. As a final note, I should point out that this approach has only been tested on Lion (10.7), although I suspect it will work just as well in earlier releases of Mac OS X. Have fun!

Previous: The /Volumes/MobileBackups directory Most recent entry

About me
My name is Daniel Stødle. I live in Tromsø, Norway, at 69.66° North. By day, I work as a researcher at the Northern Research Institute in Norway; by night, I run my company SCSC. I do most of my development on and for Mac OS X. My research is geared towards interaction with and visualization of geospatial data. Read more on my personal page.


Contact me

Recent posts

Jul 28: Running iTunes in a debugger (gdb)
Jul 25: The /Volumes/MobileBackups directory
Jul 20: FolderGlance 3.0.1 supports Lion
Mar 03: Quick tip: Speeding up Xcode compilations
Mar 02: FolderGlance 3.0
Jan 07: Making Universal Back Button work on 10.6.5 and later

Sep 03: Creating pthreads in C++ using pointers to member functions
May 31: Quickly open URLs in Terminal
May 31: Snow Leopard and automatically submitted Crash Reports
May 27: Universal Back Button released for Mac OS X
May 22: The 22 Megapixel Laptop
Feb 09: FolderGlance on MacUpdate Promo

Sep 28: FolderGlance 2.5.3 is out
Sep 21: FolderGlance 2.5.1 adds features and fixes bugs
Sep 16: FolderGlance 2.5 released!
Sep 10: Intriguing: Snow Leopard ships with the iPhone's multi-touch API built-in
Sep 03: FolderGlance and Snow Leopard
Mar 15: Fixing Keynote '08 and '09 to work with the Scripting Bridge
Feb 26: A website in an image
Feb 09: Display wall multi-touch

Feb 19: Spaces.. Spaces.. Spaces.. retires
Feb 08: How-to: Reverse engineering the Dock to fix Spaces
Jan 25: Interacting with wall-sized displays

Dec 20: Interesting Finder bug
Dec 06: Developing applications for the iPod touch (and the iPhone)
Nov 15: Spaces.. Spaces.. Spaces.. and 10.5.1
Nov 15: Thread creation using pthread_create() on Leopard
Nov 13: Spaces.. Spaces.. Spaces..
Nov 07: FolderGlance, Leopard and the More... menu
Nov 06: FolderGlance and Screen Sieve now also on Leopard!
Sep 16: Mysterious window server hangs


RSS feed

Blog frontpage