SetWindowLongPtr not seem to work

Question

I tried to subclass another window (in another process) so I injected a dll, which calls SetWindowLongPtr, but it fails and GetLastError returns 5.

BOOL APIENTRY DllMain (HANDLE hDll, DWORD dwReason, LPVOID lpReserved)
{
    switch (dwReason)
    {
        case DLL_PROCESS_ATTACH:
        {
            HWND hwnd = GetHwndProc();

            if (!(orgWndProc = (WNDPROC)SetWindowLongPtr(hwnd, GWLP_WNDPROC, (LONG_PTR)SubclassProc)))
            {
                char buf[40];
                sprintf(buf, "Error code: %d", GetLastError());
                MessageBox(hwnd, buf, "Error", MB_OK);
            }   
            break;
        }
    }
    return TRUE;
}

EDIT: Its defiantly the right PID.

EDIT 2: I was getting the wrong HWND but that is fixed now (edited the code as well) I'm no longer getting the error 5 (from GetLastError)

    HWND GetHwndProc()
{
    HWND hwnd = GetTopWindow(NULL);
    DWORD currentPID = GetCurrentProcessId();
    do
    {
        char title[256];
        if ((GetWindowText(hwnd, title, 256) > 0) && (IsWindowVisible(hwnd)))
        {
            DWORD procId;
            GetWindowThreadProcessId(hwnd, &procId);

            if (procId == currentPID)   
            {
                MessageBox(hwnd, title, "", MB_OK);
                return hwnd;
            }
        }

        hwnd = GetNextWindow(hwnd, GW_HWNDNEXT);
    } while (hwnd);
}

WNDPROC orgWndProc;
LRESULT APIENTRY SubclassProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
    switch (msg)
    {
        case WM_LBUTTONDOWN:
            MessageBox(0, "Subclass", "", 0);
            return TRUE;

        default:
            return CallWindowProc(orgWndProc, hwnd, msg, wParam, lParam);
    }
}

Thank you for reading!

Answer 1

You need to call SetWindowSubclass from the thread where the window was created, at which the message queue associated with it runs. From SetWindowSubclass reference:

Warning You cannot use the subclassing helper functions to subclass a window across threads.

In turn SetWindowLongPtr must be called from the process where the window was created. From SetWindowLongPtr reference:

Windows XP/2000: The SetWindowLongPtr function fails if the window specified by the hWnd parameter does not belong to the same process as the calling thread.

There is also the User Interface Privilege Isolation which restricts access even further.