How to tell when a transfer is complete with libssh2?

Question

I'm using libssh2 for scp file transfer which in general is working. However, I'm currently trying to read some file from a weird SSH server which doesn't support to retrieve the meta data of the file upfront. Thus I don't know how big the file really is that I want to download.

I found out that libssh2 is putting a null byte at the end of the received data if the data is complete, which is ok for text files, but will fail or lead to incertainties for binary data which itself may contain a byte 0x00.

Assume this code:

LIBSSH2_CHANNEL* ch = libssh2_scp_recv2(session, remote_file_path, NULL);

ssize_t bytes_received = 0;
bool eof_received = false;
if (ch) {
    do {
        char buf[1024] = {0};
        bytes_received = libssh2_channel_read(ch, buf, sizeof(buf));

        eof_received = (buf[bytes_received - 1] == 0x00));
        printf("Received %d bytes.\n", bytes_received);
    } while (bytes_received > 0 && !eof_received);
}

If I transfer a binary file > 1024 bytes and by coincidence the 1024th byte of the file is 0x00, the code above will think EOF is reached and stop the transfer. I also tested libssh2_channel_eof but this always gives false.

So, how can we properly know when a file is completely downloaded via SCP without knowing the expected filesize upfront?

Answer 1

the libssh2 LIBSSH2_SESSION struct has some fields used for scp receive operations. While receiving a file, the field session->scpRecv_size will contain the size of the file being transferred. There are other fields for the file's permissions and timestamps.

As you've indicated, passing a libssh2_struct_stat* to the libssh2_scp_recv2() call causes the scp receive logic to include the "-p" option in the remote scp invocation. The stat struct is eventually populated from the fields of the session struct around here. If you don't want to send "-p" to the remote scp invocation, then you should pass null as the third argument to libssh2_scp_recv2(), and get the expected file size from session->scpRecv_size.

Answer 2

The official example of scp.c shows how to do it https://github.com/libssh2/libssh2/blob/master/example/scp.c#L152C17-L152C25 .

    channel = libssh2_scp_recv2(session, scppath, &fileinfo);
    while(got < fileinfo.st_size) {
        char mem[1024];
        int amount = sizeof(mem);
        ssize_t nread;

        if((fileinfo.st_size - got) < amount) {
            amount = (int)(fileinfo.st_size - got);
        }

        nread = libssh2_channel_read(channel, mem, amount);
        if(nread > 0) {
            write(1, mem, nread);
        }
        else if(nread < 0) {
            fprintf(stderr, "libssh2_channel_read() failed: %d\n",
                    (int)nread);
            break;
        }
        got += nread;
    }