Tools, approaches for analysing proprietary data format?

Question

I need to analyse a binary data file containing raw data from a scientific instrument. A quick look in a hex viewer indicates that's probably no encryption or anything fancy: integers will probably be written as integers (but I don't know what byte order), and who knows about floating point.

I have access to a (closed source) program that can view the contents of the file. So I can see that a certain value is 74078. Actually searching for that value I'm not sure about - do I search for 00 01 21 5E, some other byte order, etc? (Hex Fiend doesn't support searching for decimal values) And how would I find a floating point number?

The software that produces these files runs on XP. I'd prefer tools that run on OSX if possible.

(Hmm, I wrote up this question, forgot to post it, then solved the problem. I guess I will write my own answer.)

Answer 1

In the end, Hex Fiend turned out to be just enough. What I was expecting to do:

1. Convert a known value into hex
2. Search for it

What I actually did:

1. Pick a random chunk of hex that looked like it might be a useful value
2. Tell Hex Fiend to display it as integer, or as float, in either little endian or big endian, until it gave a plausible looking result (ie, 45.000 is a lot more plausible than some huge integer)
3. Search for that result in the results I had from the closed source program.
4. Document it, go back to step 1. (Except that normally the next chunk wouldn't be 'random', but would follow sequentially.)

In this case there were really only three (binary) variables for how to interpret data:

1. float or integer
2. 2 bytes or 4 bytes
3. little or big endian

With more variables the task would be a lot harder. It would have been nice if Hex Fiend could search for integers/floats directly, perhaps trying out the different combinations. Perhaps other hex viewers do.

And to answer one of my original questions, 74078 turned out to be stored as 5E2101. A bit more trial and error and I would have got there. :)

UPDATE If I was doing this over, I'd use "Synalyze It!", a tool designed for exactly this purpose.