Convert bytes to a file object in python

Question

I have a small application that reads local files using:
open(diefile_path, 'r') as csv_file
open(diefile_path, 'r') as file
and also uses linecache module

I need to expand the use to files that send from a remote server.
The content that is received by the server type is bytes.

I couldn't find a lot of information about handling IOBytes type and I was wondering if there is a way that I can convert the bytes chunk to a file-like object.
My goal is to use the API is specified above (open,linecache)
I was able to convert the bytes into a string using data.decode("utf-8"),
but I can't use the methods above (open and linecache)

a small example to illustrate

data = 'b'First line\nSecond line\nThird line\n'

with open(data) as file:
    line = file.readline()
    print(line)

output:

First line
Second line
Third line

can it be done?

Answer 1

open is used to open actual files, returning a file-like object. Here, you already have the data in memory, not in a file, so you can instantiate the file-like object directly.

import io


data = b'First line\nSecond line\nThird line\n'
file = io.StringIO(data.decode())
for line in file:
    print(line.strip())

However, if what you are getting is really just a newline-separated string, you can simply split it into a list directly.

lines = data.decode().strip().split('\n')

The main difference is that the StringIO version is slightly lazier; it has a smaller memory foot print compared to the list, as it splits strings off as requested by the iterator.

Answer 2

The answer above that using StringIO would need to specify an encoding, which may cause wrong conversion.

from Python Documentation using BytesIO:

from io import BytesIO
f = BytesIO(b"some initial binary data: \x00\x01")