IOCP WSARecv always return error 10022

Question

I'm using IOCP on Windows. Previously I used method GetQueuedCompletionStatus to poll the queue and everything was fine. But when I decided to refactor the logic in the way to utilize completion routine with WSARecv call it always fails with the error WSAEINVAL (10022). This code is in thread created with CreateTread

int flags = 0;
m_iocport = CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, NULL, 0);
handle = CreateIoCompletionPort(clientSocket, m_iocport, 0, 0);
OVERLAPPED_EX *over = new OVERLAPPED_EX();
result = WSARecv(clientSocket, &over->m_wsabuf, 1, NULL, &flags, over, WorkerRoutine);

And the worker routine is empty and has following definition:

void static CALLBACK WorkerRoutine(DWORD Error, DWORD BytesTransferred, LPWSAOVERLAPPED Overlapped, DWORD InFlags) {}

When I pass NULL instead of WorkerRoutine to the WSARecv method everything works fine. But when I pass completion routine to the call it fails with the error 10022. I tried to use WorkerRoutine and &WorkerRoutine nothing helps.

the hEvent property is set to NULL in the OVERLAPPED_EX object.

Answer 1

the I/O completion port associated with file and lpCompletionRoutine this is mutually exclusive parameters. you can not use it both at once. when you do this - kernel return STATUS_INVALID_PARAMETER which is translated to WSAEINVAL. so you and must got exactly this error.

the IOCP and ApcRoutine 2 different ways for notify you about operation complete. when you use IOCP - system send packet to IOCP on complete. you need use GetQueuedCompletionStatus or NtRemoveIoCompletion to extract this packet later. and this is not "poll". from another side if you use ApcRoutine system insert apc to your thread. use two ways for notify at once and in concurrent - this is logic error and kernel correct doing when return to you STATUS_INVALID_PARAMETER in this case.

unfortunately this is not clear stated in MSDN or how minimum i can not fount this. but some research + WRK source code help understand this situation:

WSARecv internally call ZwDeviceIoControlFile and as result in kernel called IopXxxControlFile. when lpCompletionRoutine != 0 the ApcRoutine parameter for IopXxxControlFile is present and you fail exactly at this point:

    //
    // If this file has an I/O completion port associated w/it, then ensure
    // that the caller did not supply an APC routine, as the two are mutually
    // exclusive methods for I/O completion notification.
    //

    if (fileObject->CompletionContext && IopApcRoutinePresent( ApcRoutine )) {
        ObDereferenceObject( fileObject );
        return STATUS_INVALID_PARAMETER;
}

---

also you can allocate m_wsabuf in stack. valid must be only buffer to which point WSABUF

If this function is completed in an overlapped manner, it is the Winsock service provider's responsibility to capture the WSABUF structures before returning from this call. This enables applications to build stack-based WSABUF arrays pointed to by the lpBuffers parameter.

so you can place to OVERLAPPED_EX only buffer to which point WSABUF.buf but not whole WSABUF. but this already not related to your error