Problem in Intercepting Keyboard Events using Windows Hooks

Question

I'm working on creating a command-line interface (CLI) editor similar to Vim for Windows using WinAPIs. Currently, I'm focusing on implementing the logic for the Edit mode (triggered by pressing 'i') and the Command mode (activated by pressing Escape), so I know of two approaches for this problem :

1.) The first approach is to create a while loop that continuously listens for the Escape key. This loop would run indefinitely on a separate thread to avoid interfering with the program flow. but the main issue with this one is this method may lead to high CPU consumption, slowing down the program and potentially causing CPU-related issues.

2.) The second approach involves using Windows hooks. I've written a simple program to intercept keyboard events and detect the 'i' and Escape keys. I've implemented a separate variable to track where the key press originates from. But, I'm encountering issues with the editor as it doesn't seem to work properly and the cursor remains in its previous position no matter whichever key we press during it's execution. Can please someone help me in here ?. ThankYou.

Here's the code for the same :-

HANDLE hConsole;
bool editMode = true;

void moveCursorToLastRow();
void moveCursorToFirstRow();

LRESULT CALLBACK KeyboardHookProc(int nCode, WPARAM wParam, LPARAM lParam) {
    if (nCode >= 0) {
        KBDLLHOOKSTRUCT* kbStruct = (KBDLLHOOKSTRUCT*)lParam;
        DWORD vkCode = kbStruct->vkCode;
        if (vkCode == VK_ESCAPE && editMode == true) {
            editMode = false;
            moveCursorToLastRow();
        }
        else if (vkCode == 'I' && editMode == false) {
            editMode = true;
            moveCursorToFirstRow();
        }
    }
    return CallNextHookEx(NULL, nCode, wParam, lParam);
}

void moveCursorToFirstRow() {
    // Set the cursor position to the first row
    COORD cursorPosition = { 3, 2 };
    SetConsoleCursorPosition(GetStdHandle(STD_OUTPUT_HANDLE), cursorPosition);
}

void moveCursorToLastRow() {
    // Get the console screen buffer size
    CONSOLE_SCREEN_BUFFER_INFO csbi;
    GetConsoleScreenBufferInfo(GetStdHandle(STD_OUTPUT_HANDLE), &csbi);

    // Set the cursor position to the last row
    COORD cursorPosition = { 3, csbi.dwSize.Y - 1 };
    SetConsoleCursorPosition(GetStdHandle(STD_OUTPUT_HANDLE), cursorPosition);
}

int main(int argc, char* argv[]) {

    HINSTANCE hInstance = GetModuleHandle(NULL);

    HHOOK hKeyboardHook = SetWindowsHookEx(WH_KEYBOARD_LL, KeyboardHookProc, hInstance, 0);
    if (hKeyboardHook == NULL) {
        std::cerr << "Failed to install keyboard hook\n" << GetLastError();
        return 1;
    }

I tried Chatgpt for this problem but it just worsens the code, and instructed me to implemet a loop which would intercept the messages from the hooks chain, trying doing so but rather got stucked on an infinte loop and nothing got showed on the screen when tried to execute my program.

Answer 1

This is some example code that busy loops waiting for a key press. Normally, these types of loops are considered bad, but I think it is ok with this type of application. ReadConsoleInput blocks until an input record has been read, so it's not constantly looping.

Note: I just wrote this, so it is far from complete. Also, I've never written a program like this, so there may be better ways.

#include <Windows.h>
#include <string>

// Reads a key from the keyboard. Blocks until user enters a key.
KEY_EVENT_RECORD getchar(HANDLE con)
{
    DWORD number_of_events;
    INPUT_RECORD record;

    while (true)
    {
        ReadConsoleInput(con, &record, 1, &number_of_events);
        if (record.EventType == KEY_EVENT && record.Event.KeyEvent.bKeyDown)
        {
            return record.Event.KeyEvent;
        }
    }
}

// Print to console
void echo(char c, COORD& pos, HANDLE con)
{
    SetConsoleCursorPosition(con, pos);
    WriteConsole(con, &c, 1, nullptr, nullptr);
    pos.X += 1;
}
void echo(const std::string& msg, COORD& pos, HANDLE con)
{
    SetConsoleCursorPosition(con, pos);
    WriteConsole(con, msg.c_str(), msg.length(), nullptr, nullptr);
    pos.X += msg.length();
}

int main()
{
    HANDLE con_in = GetStdHandle(STD_INPUT_HANDLE);
    HANDLE con_out = GetStdHandle(STD_OUTPUT_HANDLE);
    if (con_in == INVALID_HANDLE_VALUE || con_out == INVALID_HANDLE_VALUE) return -1;

    COORD pos{ 0 };      // Position to type next char
    COORD cmd_pos{ 0 };  // Position to type next command input char

    CONSOLE_SCREEN_BUFFER_INFO csbi;
    GetConsoleScreenBufferInfo(GetStdHandle(STD_OUTPUT_HANDLE), &csbi);
    cmd_pos.Y = csbi.dwSize.Y - 1;

    // Cursor position. It can be either pos or cmd_pos.
    COORD *current = &pos;

    while (true)
    {
        KEY_EVENT_RECORD key = getchar(con_in);
        // Escape key? Enter command mode
        if (key.wVirtualKeyCode == 27) {
            current = &cmd_pos;
            echo("COMMAND: ", *current, con_out);
        }
        // New line
        else if (key.wVirtualKeyCode == 13) {
            // In text entry mode?
            if (current == &pos) {
                pos.X = 0;
                pos.Y += 1;
            }
            // In command mode?
            else {
                current = &pos;
                cmd_pos.X = 0;
                // TODO: Clear the command....
                // TODO: Execute command.....
            }
            SetConsoleCursorPosition(con_out, *current);
        }
        else {
            echo(key.uChar.AsciiChar, *current, con_out);
        }
    }
    return 0;
}

Another option would be to put the getchar function in a separate thread and use events to alert the main program that there is input available.