Breakpoints and other Debug Events

There are 3 major types of events you encounter when debugging:

  1. Breakpoints
  2. Memory violations (!!)
  3. Exceptions
buffer overflow, for example,  is probably the most well-known for of memory violation and is both a notorious headache for end-users of buggy software and a breath of fresh air for reverse engineers.
     There are 3 types of breakpoints of interest:
  1. Software Breakpoints
  2. Hardware Breakpoints
  3. Memory Breakpoints
Soft breakpoints are used specifically to halt the CPU and are the most common type of breakpoint. Soft breakpoints are sing-byte instructions that stop execution of the debugger process. In RStudio you can set a soft breakpoint simply by clicking on the left margin of a particular line or function (or by choosing “Rerun with Debugger” in the error traceback dialog). Soft breakpoints can be persistent if they remain after the CPU has executed its tasks or one-shot if they are cleared out. Soft breakpoints are associated with the interrupt 3 (INT3) event.

Using assembly language, to process a soft breakpoint the single byte instruction must be converted into an operation code (a.k.a. an opcode).

In x86 Assembly language an instruction looks like:
which tells the CPU to move the value stored in the EBX register to the EAX register. In native opcode, this would just be:
A common analogy is that x86 Assembly language is to individual opcodes what DNS is to IP addresses. Memorizing thousands of opcodes is impractical, whereas assembly language is much more manageable.

Hardware breakpoints, associated with the INT1 event, are useful when a small number of breakpoints are needed and the software can’t be modified. When registers are used in this way they’re known as debug registers.

Most CPU’s have 8 debug registers denoted as DR0 through DR7. Up to 4 of the 8 registers can be used for breakpoints and each can only set up to 4 bytes at a time. This means you can only use up to 4 hardware breakpoints at a time. DR0 to DR3 is usually reserved for the breakpoint addresses. DR4 and DR5 are reserved and DR6 is the status register (which determines the type of debugging event triggered by a breakpoint).
Unlike soft and hard breakpoints, memory breakpoints are not true breakpoints. Rather, a memory breakpoint refers to the permissions on a region or page of memory set by a debugger.
memory page is the smallest portion of memory that an OS deals with.
Examples of memory page permissions:
  1. Page execution
  2. Page read
  3. Page write
  4. Guard page
The guard page permission allows us to separate the heap and the stack and can ensure that a chunk of memory does not grow past a given boundary. It does so by  throw a one-time exception whenever accessed and then returning to its original status.