On Quora I asked a question : https://www.quora.com/What-are-some-good-tools-to-help-reverse-engineer-obscure-binary-file-formats
What tools are there to help reverse engineer obscure binary file formats?
So far the answers aren’t particularly informative. Basically hex editors. But I feel there must be something more. Here’s a comment I replied to someone there :
Well, if I try to imagine “something better”, for example, I can think of a program that I could feed a lot of different example files of the same type to, and have it compare across them to identify common sequences or patterns at particular points. (This kind of software exists for analysing databases of DNA, trying to understand the genes, for example)
I see Professional Text/Hex Editor with Binary Templates has templates for describing file formats. But I’d like to have the equivalent of a simple BNF-style grammar notation to describe formats and be given instant feedback, as I’m experimenting with a grammar, whether it fails in matching any of the files of the type I’m trying to reverse engineer.
If I know some information about the kind of data that is being represented, I’d like to be able to give my tool “hints” about it, which it can use to try to deduce what means what.
For example, I’d like to be able to say “this vector image file contains just a red triangle, that one, just a red square and that other one, just a blue square. Given that, what are the likely candidates to represent red, and what are the likely candidates to represent a square?”
Surely there must be stuff like this out there somewhere?