Downloading a file into a specific directory results in errors

Question

First, I want to clarify that I am aware questions like this have been asked before (e.g. How to resolve Node.js: "Error: ENOENT: no such file or directory"). But none of the ones I've found, seem to have a solution that fits my case. So, I am asking for solutions that would possibly be specific to my case.

I'm creating an archival tool for an art site I like. Basically I'm trying to use a URL to fetch an image (the URL is the base URL of the image), and then download it into a specific directory in my project.

I would expect fs.createWriteStream(this.file_path, {flags: "w"}) in my below code to create the file if it doesn't exist, which is kind of the point--it isn't going to exist until I fetch the image from the URL and pipe it. I have two classes:

my_project_root/lib/data/downloader.ts:

import * as https from "https";
import * as fs from "fs";
import axios from "axios";
import * as path from "path";

export class Downloader {
    private dirname: string = "";
    private artist: string = "";
    private dir_to_check: string = path.join(".", "downloaded", "artists");
    private file_path: string = path.join(".", "downloaded", "artists");
    private url: string = "";

    public constructor(dirname: string, artist: string, fileNameWithExt: string, url: string) {
        this.dirname = dirname;
        this.artist = artist;
        this.dir_to_check = path.join(this.dir_to_check, artist);
        this.file_path = path.join(dirname, this.file_path, artist, fileNameWithExt);
        this.url = url;
    }

    public async downloadImage(): Promise<boolean> {
        // Log paths for testing
        console.log(
            `
            __dirname: ${this.dirname}
            dir_to_create_if_not_exist: ${this.dir_to_check}
            file_path: ${this.file_path}
            artist: ${this.artist}
            `
        );

        // Create the directory if it doesn't exist
        const dirCreated = await this.dirCreateIfNotExists();
        if (!dirCreated) {
            return false;
        }

        try {
            // Create writer
            const writer: any = fs.createWriteStream(this.file_path, {flags: "w"});

            // File was opened
            writer.on("open", () => {
                console.log(`File opened for writing: ${this.file_path}`);
            });

            const writeStreamPromise = new Promise<boolean>((resolve, reject) => {
                // File is finished creating
                writer.on("finish", () => {
                    console.log(`Write stream finished for: ${this.file_path}`);
                    resolve(true); // Image download successful
                });
    
                // There was an error during file writing/creation
                writer.on("error", (error) => {
                    console.error(`Error during write: ${error}`);
                    resolve(false); // Image download failed
                });
            });

            // Fetch image
            const response = await axios({
                method: "GET",
                url: this.url,
                responseType: "stream",
                httpsAgent: new https.Agent({ rejectUnauthorized: false }),
                headers: {
                    "User-Agent": "my_user_agent_here",
                    "Authorization": "Basic " + btoa(`${process.env.key}:${process.env.username}`),
                },
            });

            response.data.pipe(writer);

            // Log the response status
            console.log(response.status); // Returns 200 (OK)

            // Return a promise that resolves when the write stream finishes
            return new Promise((resolve, reject) => {
                writer.on("finish", () => {
                    resolve(true); // Image download successful
                });
                writer.on("error", (error) => {
                    console.error("Error during download:", error);
                    resolve(false); // Image download failed
                });
            });
        } catch (error) {
            console.error("downloadImage() error: " + error);
            return false;
        }
    }

    private async dirCreateIfNotExists(): Promise<boolean> {
        const targetDir: string = path.join(this.dir_to_check);
        try {
            // Directory creation successful
            await fs.promises.mkdir(targetDir, {recursive: true});
            return true;
        } catch (error) {
            // Directory creation failed
            console.error("Error creating directory:", error);
            return false;
        }
    }
}

my_project_root/lib/process/download.ts: (Just showing relevant code for this one):

this.contents_of_files.forEach(async contents => {
    contents = JSON.parse(contents);
    let data: any = contents[0];
    // Do stuff
    this.id = data.id;
    this.createdAt = data.created_at;
    this.updatedAt = data.updated_at;
    this.url = data.file.url;
    this.fileExt = data.file.ext;
    this.md5 = data.file.md5;
    this.artists = data.tags.artist;

    // Upload the image/file into each artist's folder(s) it's associated with
    await this.initiateDownload(dirname); // Dirname is my drive root all the way to my project root, for example: E:\my_project\
});

My initiateDownload() method in 'download.ts':

async initiateDownload(dirname) {
    for (const artist of this.artists) {
        const file: string =`${this.md5}__${this.id}__${this.createdAt}__${this.updatedAt}.${this.fileExt}`;
        this.Downloader = new Downloader(dirname, artist, file, this.url);
        await this.Downloader.downloadImage();
    }
}

The error I'm receiving appears to have an issue with the file itself not existing yet, which is odd considering I'd expect it to create it if it doesn't exist.

The directory exists all the way to the artists folder/dir, and of course also the directories inside of that one that are named after the artist. For example: my_project_root\downloaded\artists\artist_name\ exists. If I remove the dir \artist_name\, it will be created again when the script runs. So to me, it doesn't seem like a pathing issue.

I'm receiving an ENOENT error:

Error: ENOENT: no such file or directory, open 'E:\my_project_root\downloaded\artists\artist_name\file_name.jpg'

I'm not sure why I'm getting an ENOENT error when obviously it knows exactly where to create the directories. This seems like a conflict when trying to write to a file that it thinks should exist but obviously doesn't, right?

For further clarification, my console.log() statement at the top of my downloadImage() method returns this in the console:

__dirname: E:\my_project_root
dir_to_create_if_not_exist: downloaded\artists\artist_name
file_path: E:\my_project_root\downloaded\artists\artist_name\file_name.jpg
artist: artist_name

EXTRA: I've just tried my codebase on another computer and it works just fine with no differences in code at all. Basically I just copied the project over and initialized npm for that project dir. The only difference is where on the other computer the project is located at. Is it possible this is a permissions issue with the project taking place on a secondary harddrive...?

Answer 1

So, the problem ended up being my file name:

async initiateDownload(dirname) {
    for (const artist of this.artists) {
        const file: string =`${this.md5}__${this.id}__${this.createdAt}__${this.updatedAt}.${this.fileExt}`;
        this.Downloader = new Downloader(dirname, artist, file, this.url);
        await this.Downloader.downloadImage();
    }
}

This:

const file: string =`${this.md5}__${this.id}__${this.createdAt}__${this.updatedAt}.${this.fileExt}`;

This should have been obvious to me, but for some reason it wasn't; the createdAt and updatedAt properties contained formats with characters (e.g. :) that are illegal for file names. Simply setting the file name itself as the md5 hash string resolved all issues:

const file: string = `${this.md5}.${this.fileExt}`;