Obfuscating asm code

[Simon 'corecode' Schubert]

Jonathon McKitrick wrote:
Are there any good ways to obfuscate asm code that would work in a library?  I
know there are things like encryption of binaries and such, but it seems these
really only work in the Win32 world.
What are some good ways to hide/obfuscate asm routines in a typical BSD
executable or library?
I don't know what you are doing but...

Obfuscation is not easy.  The best stuff I've seen was a combination of 
intra-command jumps (all over, every second command or such) with 
self-modifying code (close-reach for pipeline tricks and out of pipeline 
range) and doubly used commands (depending on how it was jumped to).  If 
this stuff is all over the binary it can be a really tedious work to 
figure out how it works.  Oh and sprinkle the code with function calls 
that never return (breaking common execution flow) and with 
timers/signal handlers to trick debuggers.  Basic rule is:  make 
disassembly not working by e.g. jumping into used(!) commands.  Make 
step-by-step debugging not working by exploiting pipeline tricks. 
Require every single op to be stepped by changing the execution flow all 
the time.  Do lots of this, but without pattern.  Hide single ops that 
do your actual algorithm in all the obfuscation mess.  I'd say one op 
per 100-300 other ops.  Do NOT use (return!) meaningful values! 
Interweave functionality with obfuscation all over.

You can add encryption, but that's even more advanced:  decrypt on the 
go, have different parts of the code fold and unfold all the time, try 
to prevent the attacker getting a full clear view picture.  Includes 
obfuscation of course.

But in the end the attacker will always win.  If he wants to.

cheers
  simon
--
Serve - BSD     +++  RENT this banner advert  +++    ASCII Ribbon   /"\
Work - Mac      +++  space for low $$$ NOW!1  +++      Campaign     \ /
Party Enjoy Relax   |   http://dragonflybsd.org      Against  HTML   \
Dude 2c 2 the max   !   http://golden-apple.biz       Mail + News   / \