Layout independent keys with Linux & X11

With all the talk of new display managers like Mir and Wayland rising, this is perhaps an outdated topic to discuss, but with overall documentation about X11 and Xkb keyboard handling being pretty sparse, I thought I’d note my findings down anyway.

Getting a layout independent ID for a key - the same concept as a scancode - is especially useful for games where you want to bind keys for specific finger positions like WASD without worrying if the player is using an AZERTY keyboard, or an alternate layout like dvorak or colemak.

Doing this on Linux and X11 turns out to actually be pretty easy, although not well documented.

To give some context, X11 has two indentifiers related to keys: KeySyms and KeyCodes. KeySyms are 32-bit ints which represent any symbol that might appear on a key. Each key can have multiple KeySyms, for example the Q key on a QWERTY keyboard has (at least) two: q and Q (with alt-gr on a UK keyboard I get @ too).

KeyCodes are 8-bit numbers ranging from 8 to 255 which each represent a physical key. That same Q key only has one KeyCode no matter if you hold shift or not. The mapping of KeyCodes to physical keys is implementation defined though - depending on the keyboard driver in use, the Q key could have two differing KeyCodes between two computers.

However, in modern Linux distributions the keyboard driver is almost invariably going to be evdev - so these KeyCodes can actually be used just fine as a layout independent key identifier!

Wait so you’re saying we’ve solved this already? Yup, so if that’s all you wanted to know you can just look up the KeyCode in xev and stop reading. But let’s suppose that evdev was not ubiquitous, could we still get a layout independent ID for each key? If you use a small bit of Xkb arcane magic, then the answer, as I understand it, is yes.

It turns out XKb knew this was what we wanted all along, and as part of its RMLVO system has names for every key on a standard US keyboard that are independent of what is actually printed on the keys. If you take a look at the files in /usr/share/X11/xkb/keycode/ you’ll see keys labelled with names such as “AD01”. This corresponds to the key 4 rows up (the D) and 1 across - on QWERTY it’d be that Q key again.

That’s all well and good, but how can we map this to a KeyCode or KeySym at runtime? Handily, XKB stores an array of these “AD01” style names with the array’s index corresponding to the X11 KeyCode. So it’s just a matter of iterating through this array, finding the strange name of the key you’re interested in, and noting down what the index was. An example:

#if 0
    gcc -std=c99 $0 -o xkb-key-id -lX11 && ./xkb-key-id && exit
#endif
#include <X11/XKBlib.h>
#include <stdio.h>
#include <string.h>
#define countof(x) (sizeof(x)/sizeof(*x))

// Define which keys we want to look up
enum { MY_KEY_UP, MY_KEY_DOWN, MY_KEY_LEFT, MY_KEY_RIGHT };

struct KeyInfo {
    const char* pos_name;
    int keycode;
} keys[] = {
    [MY_KEY_UP]    = { "AD02" }, // 'w' on qwerty
    [MY_KEY_DOWN]  = { "AC02" }, // 's' on qwerty
    [MY_KEY_LEFT]  = { "AC01" }, // 'a' on qwerty
    [MY_KEY_RIGHT] = { "AC03" }, // 'd' on qwerty
};

int main(void){

    // Open a handle to the X11 display server + initialize XKB
    int ev, err, maj = XkbMajorVersion, min = XkbMinorVersion, res;
    Display* dpy = XkbOpenDisplay(NULL, &ev, &err, &maj, &min, &res);
    if(!dpy) return 1;

    // Find the keycodes for the keys we're interested in.
    XkbDescPtr xkb = XkbGetKeyboard(dpy, XkbAllComponentsMask, XkbUseCoreKbd);
    for(int i = 0; i < xkb->max_key_code; ++i){
        for(int j = 0; j < countof(keys); ++j){
            if(strncmp(keys[j].pos_name, xkb->names->keys[i].name,
                    XkbKeyNameLength) == 0){
                keys[j].keycode = i;
            }
        }
    }

    // Get the state + group for XkbLookupKeySym as (sort of) described by
    // XKBProto.pdf section 2.2.2. (With probably broken wrapping?)
    // I recommend just using the xkey.state field when you get a KeyPress
    // or KeyRelease, but this example doesn't do any event handling.
    XkbStateRec xkb_state = {};
    XkbGetState(dpy, XkbUseCoreKbd, &xkb_state);
    int key_state = xkb_state.group << 13;

    // Print out the keycodes / keysyms we got
    for(int i = 0; i < countof(keys); ++i){
        KeySym sym = NoSymbol;
        XkbLookupKeySym(dpy, keys[i].keycode, key_state, NULL, &sym);
        printf("Key %d: \"%s\", keycode: %d, keysym: %s\n",
               i, keys[i].pos_name, keys[i].keycode, XKeysymToString(sym));
    }

    return 0;
}

The KeyCodes printed out should match the codes listed in the KeyCode files, and also the KeyCodes from xev. I haven’t thouroughly tested it, but I expect it should work regardless of the keyboard driver in use.

To make use of these KeyCodes you’d compare them against the xkey.key_code field of KeyPress and KeyRelease events from X11 - if they match you know which key was pressed/released.

If you want to get KeySyms from this, then you should be a bit careful - XKeycodeToKeysym will only give you the KeySym from “group 1” of the keys. This means that if a user has multiple keyboard layouts configured which they switch between, then only the KeySym from the first layout will be returned, even if they’re currently using the second.

That’s probably why the function was deprecated - you should use XLookupKeysym instead which uses the struct you get from KeyPress / KeyRelease events (i.e. &event.xkey), and takes the currently active group into account. Likewise for XkbKeycodeToKeysym which takes the state field of the same event that has the group encoded in its upper bits (You could calculate it manually if you’re careful).

If you want text input as well then use Xutf8LookupString which gives you the correct KeySym as well as correct unicode text, accounting for dead keys, compose keys and other complexities. I wrote a small example some time ago for that here.

Hopefully that sheds some light on Linux + X11’s keyboard situation. TL;DR: KeyCodes are good enough for layout independence, but there’s other fun stuff lurking in the dusty X11 headers.

Thanks for reading, If you have any comments, corrections or questions, drop me an email: alex @ this domain.