Is there a way to take convert a char array to a numeric value past a 4 byte value?

Question

For example I have:

Dir[j].FileSize = buffer1[0] << 24| buffer1[1] << 16| buffer1[2] << 8| buffer1[3]

which gives me the correct value for 0x6400 which is 25600 but 0xC800 doesnt become 51200 but -14336 instead. I really like this bitshift method because it makes sense to me but if you know another way of converting it that would be good too :)