a glob of nerdishness

I’ve been doing a lot of C++ coding at work lately, and sometimes the only “source” Xcode can show when debugging a compiled library is assembly code. Could knowing assembly language help debug C++ code? Short answer: no. There’s better techniques, and the assembly code often takes me closer to the machine (and the deadline) than I need to be in those instances.

Yet I’ve had a longtime interest in assembly code, for numerous reasons(1). Take this tiny piece of segmentation-faulting assembly:

call *8(%edx)

I had a hunch edx was a processor register (a “variable” of sorts), and sure enough Xcode’s debugger showed a 0 in the EDX register. “Call” then seems to say I’m trying to execute code from a bad function pointer that got written into EDX. But what’s with the “*8″, and does the ‘%’ mean anything special? Enter a great tutorial on AT&T Assembly Syntax, which happens to be what the GNU toolset, and therefore Xcode, uses(2). From that, we see that the percent sign is just a sigil that prefixes a register name.
The “*8(” part is a bit trickier. Under “Memory Addressing”, we see that a memory takes the form of “segment-override:signed-offset(base,index,scale)”. Don’t ask me what all that is, but it seems in our case we can simplify that down to “signed-offset(base)”. Lower down, we see that “Branch addressing using registers or memory operands must be prefixed by a ‘*’”. So it appears that this instruction says: Call the code that is located 8 past the address in the EDX register. Cool!(3)

It’s still on my to-read list, but if assembly language is interesting, you might want to check out Paul A. Carter’s PC Assembly Language free PDF book, geared towards assembly language from C. Let me know how it goes!

1. …from the days when the processor was closer, and just wanting to know how it worked, as well as wanting to write über-optimized code, modify my GPS’s firmware and make things with embedded processors…with only an assembler and raw coder-manliness! Now that Apple uses x86 on their desktop machines, that’s all the more reason to learn
2. To fully decode Assembly Language, you’ll also need a mnemonic reference for your architecture. The sig9 tutorial uses IA-32, which is has a good chance of being what you’re using.
3. Or in our case, not so cool. Since the EDX register contains 0, this would call code at address 0×8, which isn’t our code. Thankfully, the kernel detects an address this messed up and puts a quick stop to the program. However, most of those worrisome “arbitrary code execution” security holes which Microsoft was particularly good at work using similar unexpected-address calls: a cracker finds a way to a) put some machine code into memory, and b) put a “call” into the list of upcoming instructions that will run said machine code.