Why does XGrabKey return BadRequest?

Question

So I'm working on a Gtk/X11/Linux app that does screen capture to .gif and one of the methods of stopping the capture is a key press (Esc, Space or End). You can also use a timeout. However to implement the key press to end capture I have to be able to grab the key such that I can get an event even though my window doesn't have focus (it's actually invisible during capture). I believe XGrabKey is the right X11 function for this task:

Window w = Gtk::gdk_x11_drawable_get_xid(Gtk::gtk_widget_get_window(Handle()));
KeyCode kc = XKeysymToKeycode(Gtk::gdk_display, HotKeyCode);
int r = XGrabKey(   Gtk::gdk_display,
                    kc,
                    0               /* modifiers */,
                    w               /* grab_window */,
                    TRUE            /* owner_events */,
                    GrabModeAsync   /* pointer_mode */,
                    GrabModeAsync   /* keyboard_mode */);
printf("XGrabKey(%p, 0x%x/%x)=%i\n", w, HotKeyCode, kc, r);

Where 'HotKeyCode' is say XK_Escape or something e.g.:

XGrabKey(0x3e00003, 0xff1b/9)=1

XGrabKey is returning '1' or BadRequest. What am I doing wrong here?

FYI the actual Xorg Xserver code in question appears to be here.

Edit: The latest incarnation of the code is:

int x_err_callback(Display *d, XErrorEvent *e)
{
    char msg[256];

    XGetErrorText(d, e->error_code, msg, sizeof(msg));

    printf("X11Error %d (%s): request %d.%d\n",
        e->error_code, msg, e->request_code,
        e->minor_code);

    return 0;
}

Gtk::GdkFilterReturn key_filter(Gtk::GdkXEvent *gdk_xevent,
                                Gtk::GdkEvent *event,
                                Gtk::gpointer data)
{
    XKeyEvent *xevent = gdk_xevent;
    if (xevent->type == KeyPress)
    {
        int key = ((XKeyEvent *)gdk_xevent)->keycode;
        int keysym = XKeycodeToKeysym(Gtk::gdk_display, key, 0);

        printf("caught keysym %i\n", keysym);

        switch (keysym)
        {
            case 1: // your_keysym
                // your key handler code
                break;
        }
    }

    return Gtk::GDK_FILTER_CONTINUE;
}


Gtk::GdkWindow *Root = Gtk::gdk_get_default_root_window();
KeyCode kc = XKeysymToKeycode(Gtk::gdk_display, HotKeyCode);

XSetErrorHandler(x_err_callback);

int r = XGrabKey(   Gtk::gdk_display,
                    kc,
                    AnyModifier /* modifiers */,
                    GDK_WINDOW_XWINDOW(Root) /* grab_window */,
                    TRUE            /* owner_events */,
                    GrabModeAsync   /* pointer_mode */,
                    GrabModeSync    /* keyboard_mode */);

Gtk::gdk_window_set_events(Root,
    (Gtk::GdkEventMask)
    (Gtk::GDK_KEY_PRESS_MASK |
    Gtk::GDK_KEY_RELEASE_MASK));
Gtk::gdk_window_add_filter(NULL, key_filter, this);

AnyModifier actually results in an error. '0' doesn't. I know about the NumLock issue...

Answer 1

A return value of 1 does not mean that a BadRequest error occured. Xlib handles errors via an error handler, and the function will always return 1, if it returns at all.

Your code does not work because you have to do the XGrabKey on the root window (GetDefaultRootWindow(Gtk::gdk_display)). Here's a pure Xlib demo:

#include <X11/Xlib.h>
#include <X11/keysym.h>
#include <stdio.h>

int main() {
    Display *d = XOpenDisplay(0);
    Window root = DefaultRootWindow(d);
    int keycode = XKeysymToKeycode(d, XK_BackSpace);

    int rv = XGrabKey(d, keycode, AnyModifier, root, 1, GrabModeAsync, GrabModeAsync);
    printf("XGrabKey returned %d\n", rv);

    XEvent evt;
    while(1) {
        XNextEvent(d, &evt);
        printf("Got event %d\n", evt.type);
    }
}

To then capture the X11 events from GTK use gdk_window_add_filter on a NULL or on the root window and a GdkFilterFunc that processes the events associated with your global hotkey:

#include <X11/Xlib.h>
#include <X11/keysym.h>
#include <gtk/gtk.h>
#include <gdk/gdkx.h>
#include <stdio.h>

GdkFilterReturn filter(GdkXEvent *xevent, GdkEvent *event, gpointer data) {
    XKeyEvent *ev = (XKeyEvent *)xevent;
    if(ev->type == 2) {
        printf("Backspace hit.\n");
    }

    return GDK_FILTER_CONTINUE;
}

int main(int argc, char *argv[]) {
    gtk_init(&argc, &argv);

    GdkScreen *scr = gdk_screen_get_default();
    GdkWindow *groot = gdk_screen_get_root_window(scr);
    gdk_window_set_events(groot, GDK_KEY_PRESS_MASK);
    gdk_window_add_filter(groot, filter, NULL);

    Display *d = gdk_x11_get_default_xdisplay();
    Window root = GDK_WINDOW_XID(groot);
    int keycode = XKeysymToKeycode(d, XK_BackSpace);
    XGrabKey(d, keycode, AnyModifier, root, 1, GrabModeAsync, GrabModeAsync);

    gtk_main();
}

As a side note, a modifier mask of 0 means that no modifiers must be enabled, even those that would not modify the meaning of a key. A grab on the letter "A" with a 0 modifier would not match NumLock + A. That's why I used AnyModifer.