iTunes is one of a few Apple-provided applications that prevent you from attaching a debugger. This can be problematic if you are, say, developing an iTunes visualizer. iTunes accomplishes this by calling ptrace(PT_DENY_ATTACH, 0, 0, 0), which prevents future attempts at tracing the process. Debuggers like gdb use ptrace to control the process they debug. Luckily, there's a way to intercept the ptrace-call before it completes, simply by starting iTunes in the debugger. Let's see what happens when we run iTunes from gdb in Terminal, without doing anything:
[#] cd /Applications/iTunes.app/Contents/MacOS/ [#] gdb ./iTunes gdb shows some license info (gdb) run Starting program: /Applications/iTunes.app/Contents/MacOS/iTunes Reading symbols for shared libraries ... done Program exited with code 055.
Not quite what we want. Let's see what happens when we put a breakpoint on ptrace before starting iTunes:
(gdb) br ptrace Breakpoint 1 at 0x7fff90592d14 (gdb) run Starting program: /Applications/iTunes.app/Contents/MacOS/iTunes Reading symbols for shared libraries . done Breakpoint 1, 0x00007fff90592d14 in ptrace () (gdb)
We're stopped at the entry to the ptrace() function. Now, at this point we have two options: We can force the function to return, setting its return value to 0 to indicate no error, or we can modify the arguments to the ptrace call. Either way works, but this post will demonstrate the latter approach.
We know that iTunes calls ptrace with PT_DENY_ATTACH as its argument. Looking at the ptrace header file, we discover that PT_DENY_ATTACH corresponds to the value 31:
From /usr/include/sys/ptrace.h: #define PT_DENY_ATTACH 31
Now, let's look at the contents of the registers:
(gdb) info registers rax 0x24a0 9376 rbx 0x7fff5fc35120 140734800023840 rcx 0x0 0 rdx 0x0 0 rsi 0x24a0 9376 rdi 0x1f 31 Rest of register contents snipped
The register rdi looks promising. Let's try changing it to the value matching PT_DETACH:
(gdb) set $rdi=11 (gdb) info registers rax 0x24a0 9376 rbx 0x7fff5fc35120 140734800023840 rcx 0x0 0 rdx 0x0 0 rsi 0x24a0 9376 rdi 0xb 11 (gdb) continue Continuing. Reading symbols for shared libraries . done Symbols keep loading, and iTunes launches. And we're still attached!
So that seemed to work brilliantly. At this point, you can use the debugger as normal, pausing iTunes when you need to, getting backtraces, and catch errors in your visualizer. The only caveat is that detaching iTunes using gdb will probably hang, since we replaced PT_DENY_ATTACH with PT_DETACH earlier. This can be worked around by using the "return immediately" approach I alluded to earlier; doing this will be left as an exercise for the reader. To make gdb exit, you will probably have to kill it. As a final note, I should point out that this approach has only been tested on Lion (10.7), although I suspect it will work just as well in earlier releases of Mac OS X. Have fun!
Comments by Disqus
Spaces.. Spaces.. Spaces.. (Retired)
Jul 28: Running iTunes in a debugger (gdb)
Jul 25: The /Volumes/MobileBackups directory
Jul 20: FolderGlance 3.0.1 supports Lion
Mar 03: Quick tip: Speeding up Xcode compilations
Mar 02: FolderGlance 3.0
Jan 07: Making Universal Back Button work on 10.6.5 and later
Sep 03: Creating pthreads in C++ using pointers to member functions
May 31: Quickly open URLs in Terminal
May 31: Snow Leopard and automatically submitted Crash Reports
May 27: Universal Back Button released for Mac OS X
May 22: The 22 Megapixel Laptop
Feb 09: FolderGlance on MacUpdate Promo
Sep 28: FolderGlance 2.5.3 is out
Sep 21: FolderGlance 2.5.1 adds features and fixes bugs
Sep 16: FolderGlance 2.5 released!
Sep 10: Intriguing: Snow Leopard ships with the iPhone's multi-touch API built-in
Sep 03: FolderGlance and Snow Leopard
Mar 15: Fixing Keynote '08 and '09 to work with the Scripting Bridge
Feb 26: A website in an image
Feb 09: Display wall multi-touch
Feb 19: Spaces.. Spaces.. Spaces.. retires
Feb 08: How-to: Reverse engineering the Dock to fix Spaces
Jan 25: Interacting with wall-sized displays
Dec 20: Interesting Finder bug
Dec 06: Developing applications for the iPod touch (and the iPhone)
Nov 15: Spaces.. Spaces.. Spaces.. and 10.5.1
Nov 15: Thread creation using pthread_create() on Leopard
Nov 13: Spaces.. Spaces.. Spaces..
Nov 07: FolderGlance, Leopard and the More... menu
Nov 06: FolderGlance and Screen Sieve now also on Leopard!
Sep 16: Mysterious window server hangs