Tools for Reverse Engineering Binary File Formats

On Quora I asked a question : https://www.quora.com/What-are-some-good-tools-to-help-reverse-engineer-obscure-binary-file-formats

What tools are there to help reverse engineer obscure binary file formats?

So far the answers aren’t particularly informative. Basically hex editors. But I feel there must be something more. Here’s a comment I replied to someone there :

Well, if I try to imagine “something better”, for example, I can think of a program that I could feed a lot of different example files of the same type to, and have it compare across them to identify common sequences or patterns at particular points. (This kind of software exists for analysing databases of DNA, trying to understand the genes, for example)

I see Professional Text/Hex Editor with Binary Templates has templates for describing file formats. But I’d like to have the equivalent of a simple BNF-style grammar notation to describe formats and be given instant feedback, as I’m experimenting with a grammar, whether it fails in matching any of the files of the type I’m trying to reverse engineer.

If I know some information about the kind of data that is being represented, I’d like to be able to give my tool “hints” about it, which it can use to try to deduce what means what.

For example, I’d like to be able to say “this vector image file contains just a red triangle, that one, just a red square and that other one, just a blue square. Given that, what are the likely candidates to represent red, and what are the likely candidates to represent a square?”

Surely there must be stuff like this out there somewhere?

Join the Conversation

2 Comments

  1. Sorry, Github has no contact mechanism, I don’t want to comment in Adrian’s fdupes repo, and I didn’t see private contact info on this site. I saw your “Feature Request : Ignore files by size” issue in the fdupes issue tracker and I wanted to let you know about my fork called jdupes. It has various exclusion by size parameters with -X that I think will fit your needs. I also saw that you mentioned JavaScript and I was wondering if you’d find the JSON output feature I just added to be useful. Please feel free to give it a try. It is also available in most Linux distribution package repositories (I know for sure that all Debian-based ones and Arch Linux have it.) If you need a binary package, I can build and upload one for you.
    https://github.com/jbruchon/jdupes
    Hope this helps, and again, sorry I couldn’t find a better contact method for you.

    1. Thanks.
      No problem contacting me like this. I’ll take a look at jdupes.
      Cheers

Leave a comment