{"id":1363,"date":"2019-06-04T15:03:21","date_gmt":"2019-06-04T15:03:21","guid":{"rendered":"http:\/\/sdi.thoughtstorms.info\/?p=1363"},"modified":"2019-06-04T15:03:21","modified_gmt":"2019-06-04T15:03:21","slug":"tools-for-reverse-engineering-binary-file-formats","status":"publish","type":"post","link":"https:\/\/sdi.thoughtstorms.info\/?p=1363","title":{"rendered":"Tools for Reverse Engineering Binary File Formats"},"content":{"rendered":"\n<p>On Quora I asked a question : <a href=\"https:\/\/www.quora.com\/What-are-some-good-tools-to-help-reverse-engineer-obscure-binary-file-formats\">https:\/\/www.quora.com\/What-are-some-good-tools-to-help-reverse-engineer-obscure-binary-file-formats<\/a> <\/p>\n\n\n<p>What tools are there to help reverse engineer obscure binary file formats? <\/p>\n\n\n<p>So far the answers aren&#8217;t particularly informative. Basically hex editors. But I feel there must be something more. Here&#8217;s a comment I replied to someone there :<\/p>\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>Well,  if I try to imagine \u201csomething better\u201d, for example, I can think of a  program that I could feed a lot of different example files of the same  type to, and have it compare across them to identify common sequences or  patterns at particular points. (This kind of software exists for  analysing databases of DNA, trying to understand the genes, for example)<\/p><p>I see <a rel=\"noreferrer noopener\" href=\"http:\/\/www.sweetscape.com\/010editor\/\" target=\"_blank\">Professional Text\/Hex Editor with Binary Templates<\/a>  has templates for describing file formats. But I\u2019d like to have the  equivalent of a simple BNF-style grammar notation to describe formats  and be given instant feedback, as I\u2019m experimenting with a grammar,  whether it fails in matching any of the files of the type I\u2019m trying to  reverse engineer.<\/p><p>If  I know some information about the kind of data that is being  represented, I\u2019d like to be able to give my tool \u201chints\u201d about it, which  it can use to try to deduce what means what.<\/p><p>For  example, I\u2019d like to be able to say \u201cthis vector image file contains  just a red triangle, that one, just a red square and that other one,  just a blue square. Given that, what are the likely candidates to  represent red, and what are the likely candidates to represent a  square?\u201d<\/p><\/blockquote>\n\n\n<p>Surely there must be stuff like this out there somewhere?<br \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>On Quora I asked a question : https:\/\/www.quora.com\/What-are-some-good-tools-to-help-reverse-engineer-obscure-binary-file-formats What tools are there to help reverse engineer obscure binary file formats? So far the answers aren&#8217;t particularly informative. Basically hex editors. But I feel there must be something more. Here&#8217;s a comment I replied to someone there : Well, if I try to imagine \u201csomething better\u201d, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-1363","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/posts\/1363","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1363"}],"version-history":[{"count":0,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=\/wp\/v2\/posts\/1363\/revisions"}],"wp:attachment":[{"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1363"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1363"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sdi.thoughtstorms.info\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1363"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}