Python: Can't create a binary file with a "with open(xxxxx)" line, using an os.path generated path (single backslash passed as double?)

Question

I'm trying to download a video from vimeo (with this code https://github.com/moettle/hidden-vimeo-downloader, with some modified info for the request session info), and in the part of writing the file, I get an [Errno 22] message, as if the with open(PATH, "wb") read the backslashes as double backslashes

i.e: if PATH is something like "folder\filename.txt", the terminal's output shows OSError: [Errno 22] Invalid argument: 'folder\filename.txt'

This is my code:

    r = requests.get(current_segement_url)
    v_seg_file = os.path.join(out_video ,"video-" + segment_base_name + str(i) + segment_base_ext)

# Just to print out the actual variable's content
    print(v_seg_file)

    with open(v_seg_file, "wb") as f:
        f.write(r.content)

(i've separated the path to v_seg_file to see if the .encode() method or playing with replacements or raw strings could work, but it doesn't.)

The current output is:

segments_video_8cf150a5-2ce1-4adc-bb37-111111111111\video-segment-0.m4s?r=11111111dHJhbDE%3D
Traceback (most recent call last):
File "C:\Users\XXXXX\.......\hidden-vimeo-downloader\vimeo-downloader.py", line 78, in <module>
with open(v_seg_file, "wb") as f:
         ^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 22] Invalid argument: 'segments_video_8cf150a5-2ce1-4adc-bb37-111111111111\\video-segment-0.m4s?r=11111111dHJhbDE%3D'   

(the actual file path was modified here, but it's generated automatically by the code)

The question is: Why can't I create the file? Some lines above, the directory is created if it doesn't exist. I also tried writing in "wb", "wb+", "w+b" without success

I think the problem is that with open( here somehow transforms a single backslash into two? I mean, the printed output of v_seg_file is OK, which has been created with os.path.join (that should be OK)

I also tried another script to test this writing statements just as a double check, and it works fine for non-binary writing.

import os

out_video = "folder"
segment_base_name = "xxx"
segment_base_ext = "yyy"
i = 0

v_seg_file = os.path.join(out_video ,"video-" + segment_base_name + str(i) + segment_base_ext)

print(v_seg_file)

if not os.path.exists(out_video):
    os.makedirs(out_video)
    print("Directory created successfully!")
else:
    print("Directory already exists!")

with open(v_seg_file, "w") as f:
    f.write("file written")

(result: the file folder\video-xxx0yyy was created without issues, and consists of only one line of text "file written")

Could this be a platform issue with the BINARY writing? I'm running this code locally in a Windows machine, but yesterday i run it in Github codespaces (with another video URL) and worked fine.

Answer 1

Chances are, codespaces is running Linux, hence why the file creation worked there. Windows and *nix have different rules regarding what characters are allowed in a filename.

Please refer to this SO answer for more specifics.

As you’re on Windows, there are many disallowed characters, specifically for your case, a question mark. The file being binary vs plaintext, has no bearing here.

The fix:

You’ll need to clean the filename before attempting to create the file on Win.

Tip: The Python str.translate method is very useful for making multiple replacements in a string.

Answer 2

Use Raw string. example:

file path = r".\your-path"

Put 'r' in front of your directory.

Answer 3

If your path is folder\filename.txt then you'll need either r"folder\filename.txt" or "folder\\filename.txt". This is because \f is interpreted as formfeed (0x0C). Furthermore, the directory "folder" will have to exist in your current working directory and you'll need permission to write in that directory

Answer 4

Ensure your path uses either raw strings (rpath) or replaces backslashes with forward slashes (folder/filename.txt). In python, backslashes are escape characters, often leading to issues like yours when used in file paths. Also, check your path and filename for any unsupported characters, as they can trigger an Errno 22. If modifying the path doesn't help, manually print and verify the path for accuracy or potential hidden characters.