Remove 64 Bytes every 2048 bytes in Binary

Question

I'm at a loose end here, it seems like such a simple problem so I'm hoping there is a simple answer!

I have a binary (approx 35m) which has 64 bytes of padded data every 2048 bytes starting at offset 1536 - I just want to remove this padding.

The first occurrence is 1536, then 3648,5760,7872,etc

(2112 bytes - 64 bytes of dummy data = 2048)

I've tried bvi,bbe,hexdump+sed+xxd and I'm clearly missing something.

Thanks in advance,

Answer 1

You didn't show any code, so I presume you need help wrapping your head around the algorithm. It's actually quite simple:

1. While you haven't reached the EOF of STDIN,
   1. Read 2112 bytes from STDIN
   2. From the bytes read, remove the 64 bytes starting at position 1536.
   3. Print the remaining 2048 bytes to STDOUT.

In Perl,

binmode(STDIN);
binmode(STDOUT);
while (1) {
   my $rv = read(STDIN, my $rec, 2112);
   die $! if !defined($rv);
   last if !$rv;

   substr($rec, 1536, 64, '');

   print($rec)
      or die $!;
}

Answer 2

If you want to use Perl:

Open the file with the :raw layer. We don't want :utf8 or :crlf translation.

Then, we can seek to the positions we are interested in, and can read a few bytes

my $size = -s $filename;
open my $fh, "<:raw", $filename;
for (seek($fh, 1536, 0) ; tell($fh) + 2048 < $size ; seek($fh, 2048 - 64, 1)) {
  read $fh, my $buffer, 64;
  ...;
}

Read

• perldoc -f tell
• perldoc -f seek
• perldoc -f read

for further information