Grpc C++: How to wait until a unary request has been sent?

Question

I'm writing a wrapper around gRPC unary calls, but I'm having an issue: let's say I have a ClientAsyncResponseReader object which is created and starts a request like so

response_reader_ = std::unique_ptr<grpc::ClientAsyncResponseReader<ResponseType>>(
        grpc::internal::ClientAsyncResponseReaderFactory<ResponseType>::Create(
            channel.get(), completion_queue, rpc_method, &client_context_, request, true
        )
);
response_reader_->Finish(
    response_sharedptr_.get(), status_sharedptr_.get(), static_cast<void*>(some_tag)
);
// Set a breakpoint here

where all of the arguments are valid.

I was under the impression that when the Finish call returned, the request object was guaranteed to have been sent out over the wire. However by setting a breakpoint after that Finish() call (in the client program, to be clear) and inspecting my server's logs, I've discovered that the server does not log the request until after I resume from the breakpoint.

This would seem to indicate that there's something else I need to wait on in order to ensure that the request is really sent out: and moreover, that the thread executing the code above still has some sort of role in sending out the request which appears post-breakpoint.

Of course, perhaps my assumptions are wrong and the server isn't logging the request as soon as it comes in. If not though, then clearly I don't understand gRPC's semantics as well as I should, so I was hoping for some more experienced insight.

You can see the code for my unary call abstraction here. It should be sufficient, but if anything else is required I'm happy to provide it.

EDIT: The plot thickens. After setting a breakpoint on the server's handler for the incoming requests, it looks like the call to Finish generally does "ensure" that the request has been sent out: except for the first request sent by the process. I guess that there is some state maintained either in grpc::channel or maybe even in grpc::completion_queue which is delaying the initial request

Answer 1

From the documentation

response_reader_ = std::unique_ptr<grpc::ClientAsyncResponseReader<ResponseType>>(
        grpc::internal::ClientAsyncResponseReaderFactory<ResponseType>::Create(
            channel.get(), completion_queue, rpc_method, &client_context_, request, true
        )
);

This will start a call and write the request out (start=true). This function does not have a tag parameter. So there is no way the completion queue can notify when the call start is finished. Calling an RPC method is a bit complicated, it basically involves creating the network packet and putting it in the wire. It can fail if there is a transient failure of the transport or the channel completely gone or the user did something stupid. Another thing, why we need the tag notification is that the completion queue is really a contention point. All RPC objects talk to this, it can happen completion queue is not free and the request is still pending.

response_reader_->Finish(
    response_sharedptr_.get(), status_sharedptr_.get(), static_cast<void*>(some_tag)

This one will request the RPC runtime to receive the server's response. The output is when the server response arrives, then the completion queue will notify the client. At this point. we assume that there is no error on the client side, everything okay and the request is already in flight. So the status of Finish call will never be false for unary rpc.

This would seem to indicate that there's something else I need to wait on in order to ensure that the request is really sent out: and moreover, that the thread executing the code above still has some sort of role in sending out the request which appears post-breakpoint.

Perhaps, you want to reuse the request object(I did some experiments on that). For me, I keep the request object in memory till the response arrives. There is no way to guarantee that the request object won't be required after the create call.