How can you read different sized bit values from a file?

Question

I'm reading a bunch of bit values from a text file which are in binary from because I stored them using fwrite. The problem is that the first value in the file is 5 bytes in size and the next 4800 values are 2 bytes in size. So when I try to cycle through the file and read the values it will give me the wrong results because my program does not know that it should take 5 bytes the first time and then 2 bytes the remaining 4800 times.

Here is how I'm cycling through the file:

long lSize;
unsigned short * buffer;
size_t result;

pFile = fOpen("dataValues.txt", "rb");

lSize = ftell(pFile);

buffer = (unsigned short *) malloc (sizeof(unsigned short)*lSize);    
size_t count = lSize/sizeof(short);



for(size_t i = 0; i < count; ++i)
{
   result = fread(buffer+i, sizeof(unsigned short), 1, pFile);
   print("%u\n", buffer[i]);
 }

I'm pretty sure I'm going to need to change my fread statement because the first value is of type time_t so I'll probably need a statement that looks like this:

result = fread(buffer+i, sizeof(time_t), 1, pFile);

However, this did not work work when I tried it and I think it's because I am not changing the starting position properly. I think that while I do read 5 bytes worth of data, I don't move the starting position enough.

Does anyone here have a good understanding of fread? Can you please let me know what I can change to make my program accomplish what I need.

EDIT:

This is how I'm writing to the file.

fwrite(&timer, sizeof(timer), 1, pFile);
fwrite(ptr, sizeof(unsigned short), rawData.size(), pFile);

EDIT2:

I tried to read the file using ifstream

int main()
{
    time_t x;
    ifstream infile;
    infile.open("binaryValues.txt", ios::binary | ios::in);
    infile.read((char *) &x, sizeof(x));
    return 0;
}

However, now it doesn't compile and just give me a bunch of undefined reference to errors to code that I don't even have written.

Answer 1

You can't. Streams are byte, almost always octet (8 bit byte) oriented.

You can easily enough build a bit-oriented stream on top of that. You just keep a few bytes in a buffer and keep track of which bit is current. Watch out for getting the last few bits, and attempts to mix byte access with bit access.

Untested but this is the general idea.

  struct bitstream
  {
      unsigned long long rack;  // 64 bits rack
      FILE *fp;                 // file opened for reading
      int rackpos;              // 0 - 63, poisition of bits read.
  }

  int getbits(struct bitstream *bs, int Nbits)
  {
     unsigned long long mask = 0x8000 0000 0000 0000;
     int answer = 0;

     while(bs->rackpos > 8)
     {
        bs->rack <<= 8;
        bs->rack |= fgetc(bs->fp);
        bs->rackpos -= 8;
     }
     mask >>= bs->rackpos;
     for(i=0;i<Nbits;i++)
     {
        answer <<= 1;
        answer |= bs->rack & mask;
        mask >>= 1;
     }
     bs->rackpos += Nbits;

     return answer;
  }

You need to decide how you know when the stream is terminated. As is you'll corrupt the last few bits with the EOF read by fgetc().

Answer 2

I don't see the problem:

uint8_t five_byte_buffer[5];
uint8_t two_byte_buffer[2];
//...
ifstream my_file(/*...*/);
my_file.read(&five_byte_buffer[0], 5);
my_file.read(&two_byte_buffer[0], 2);

So, what is your specific issue?

Edit 1: Reading in a loop

while (my_file.read(&five_byte_buffer[0], 5))
{
  my_file.read(&two_byte_buffer[0], 5);
  Process_Data();
}