Lua debugger, hooks, yield and resume

Question

Im attempting to make a simple debugger for my game engine to allow to to step line by line through a lua script loaded from disk. All done via the c api.

I can setup a hook to trigger every line, or every x instructions etc. I can also create thread/coroutines to allow yielding which can return control flow back to c.

The issue comes when trying to combine these. The lua script should not have to invoke anything special for this. is calling lua_yield not allowed from a hook function?

Here I create a new lua state, a new lua thread, set a hook and run script.lua using pcallk

lua_State* L = luaL_newstate();
luaL_openlibs(L);
lua_State* T = lua_newthread(L);
lua_sethook(T, &hookfunc, LUA_HOOKLINE, 1); // Run a hook to trigger every line
luaL_loadfile(T, "script.lua");
int nres; // throwaway, unused
int status = lua_resume(T, 0, 0, &nres); // Run the script so that it can yield

The hook function simply calls yield.

lua_yield(L, 0);

This doesn't work as expected, is this legal? did I miss something in documentation.

I've looked at:

• PIL 4th edition.
• The lua.org documentation
• This sandbox page
• And an old email thread i cannot find the link to

I will reiterate this is to be done in the c api, the lua script itself doesn't invoke anything

Answer 1

You say it doesn't work, but you don't specify what you're trying to achieve. Yielding from debug hook is not going to work (you'll get "attempt to yield across a C-call boundary" error message) because there is nothing to yield to; you haven't resumed any code to allow yield to be executed. What you can do from the debug hook is to resume some other code; this is the way some of Lua debuggers work (for example, MobDebug and RemDebug).

You may check this SO question, as it appears to describe a similar goal and one of the answers suggests an extension (yieldhook) that may be relevant to your question.

Answer 2

So it turns out my main mistake was using LUA_HOOKLINE instead of LUA_MASKLINE. I needed to use lua_resume instead of lua_pcallk. Making the thread/coroutine protected is another task. Here's a tiny working example where lua returns control back to c after every line of execution.

The example below (assuming no errors are raised) will print Executing line: 1 2... 3... and so on as the script executes. and at any moment between line you have full control in c again.

void hookfunc(lua_State* L, lua_Debug* ar)
{
lua_getinfo(L, "l", ar); // Get ar->currentline
printf("Executing line: %d\n", ar->currentline);
lua_yield(L, 0); // Yield, only works when using `lua_resume` and with 0 return values
}

int main()
{
// Create state
lua_State* L = luaL_newstate();
luaL_openlibs(L);

// Create thread/coroutine
lua_State* L1 = lua_newthread(L);

// Set hook
lua_sethook(L1, &hookfunc, LUA_MASKLINE, 1);

// Load lua script
luaL_loadfile(L1, "script.lua");

// Execute the thread until it either completes or raises an error
do
{
    int nres; // Number of return values, should be 0 when using inside hooks
    status = lua_resume(L1, 0, 0, &nres); // Start/resume thread
    // The thread may be yielded, completed or have an error, but you get full control back in c, after every line of lua execution

}
while (status == LUA_YIELD);

lua_close(L);
}