Why does FtpWebRequest return an empty stream for this existing directory?

Question

I'm not sure why am I getting this result. I'm running this on a Linux server. (It's my small web site's shared web hosting account.)

The files are grouped as follows:

and the Another Dir has one file inside:

So I'm trying to retrieve the contents of the badname directory inside 1somelongdir1234567 directory that doesn't exist on the server, using this code:

try
{
    FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create(
        "ftp://server12.some-domain.com/public_html/1somelongdir1234567/badname");
    ftpRequest.EnableSsl = true;
    ftpRequest.Credentials = new NetworkCredential("user", "password");
    ftpRequest.KeepAlive = true;
    ftpRequest.Timeout = -1;

    ftpRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;

    using (FtpWebResponse response1 = (FtpWebResponse)ftpRequest.GetResponse())
    {
        //*****BEGIN OF EDIT*****
        Console.WriteLine(response1.StatusDescription);
        Console.WriteLine(response1.StatusCode);
        //*****END OF EDIT*****

        using (StreamReader streamReader = new StreamReader(response1.GetResponseStream()))
        {
            List<string> arrList = new List<string>();

            for (; ; )
            {
                string line = streamReader.ReadLine();

                //I get to here, where `line` is null????

                if (string.IsNullOrEmpty(line))
                    break;

                arrList.Add(line);

                //*****BEGIN OF EDIT*****
                Console.WriteLine(line);
                //*****END OF EDIT*****
            }
        }
    }
}
catch (Exception ex)
{
    Console.WriteLine(ex.Message);
}

So as you see, there's no such folder badname but instead of throwing an exception, my ftpRequest.GetResponse() succeeds and then streamReader.ReadLine() returns null like I showed in the code above.

More strangely, if I provide an actual directory as such:

    FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create(
        "ftp://server12.some-domain.com/public_html/1somelongdir1234567/Another%20Dir");

the streamReader.ReadLine() still returns null.

Can someone explain why?

---

Edit: OK, guys, I updated the code above to retrieve the status code. I'm still puzzled though.

First, here's three values of connection URI and the response/output that I'm getting:

Example 1:

//Existing folder
"ftp://server12.some-domain.com/public_html/1somelongdir1234567"

Output:

150 Accepted data connection

OpeningData
drwxr-xr-x    3 username   username         4096 Sep  5 05:51 .
drwxr-x---  118 username   99               4096 Sep  5 05:54 ..
drwxr-xr-x    2 username   username         4096 Sep  5 05:52 Another Dir
-rw-r--r--    1 username   username           11 Sep  5 05:51 test123.txt

Example 2:

//Another existing folder
"ftp://server12.some-domain.com/public_html/1somelongdir1234567/Another%20Dir"

Output:

150 Accepted data connection

OpeningData

Example 3:

//Nonexistent folder
"ftp://server12.some-domain.com/public_html/1somelongdir1234567/SomeBogusName"

Output:

150 Accepted data connection

OpeningData

So why is it giving me the same result for example 2 as I'm getting for 3?

As for what ftp server it is, I wasn't able to see it in the network logs nor in FtpWebRequest itself. Here's what I was able to get from Microsoft Network Monitor:

Welcome to Pure-FTPd [privsep] [TLS] ----------..
220-You are user number 4 of 50 allowed... 220-Local time is now 13:14. Server port: 21...
220-This is a private system - No anonymous login..
220-IPv6 connections are also welcome on this server...
220 You will be disconnected after 15 minutes of inactivity...

Answer 1

The URL for the ListDirectory/ListDirectoryDetails method should end with a slash, in general.

Without a slash, results tend to be uncertain.

WebRequest.Create("ftp://example.com/public_html/1somelongdir1234567/Another%20Dir/");

Answer 2

As stuartd noted, you get an FTP status code on the response, As far as I know, because of the way FTP commands work, you will never get an exception when a server fails to execute a command you requested. The server will instead just tell you it failed through the FTP response code.

One thing you have to realize is that FTP has no standards for the text it sends back, because it was never designed for its responses to be machine-interpreted. If you got some time, look up the code of FileZilla, and check their class for interpreting the file listings (directorylistingparser.cpp). There are dozens of ways to interpret it, and the application will just try them all until one works. Because, unless you know what kind of FTP you're connecting to, that's the only way to do it.

So if that specific server decides to send an empty string back if you request a nonexistent directory, that's just how their FTP implementation does it, and it's the client's problem to figure out what that means.

If you can't figure out it's a failure from the FTP response code, you can just test if it returns something different for an existing but empty folder. If so, you can easily distinguish between that and this empty response, and treat the empty response as "not found" error in your program.