How can I extend scrapy-splash 'render.html' endpoint with some lua code?

Question

So, I want to use scrapy-splash with endpoint='execute' but mimic endpoint='render.html' with my Lua extra code. But I couldn't find an example how this should be done so that the Lua code that will be sent with the request will be the same as the Lua (I expect) used to intercept an HTML with all it's parameters. Any idea?

Answer 1

As far as I could see this is not supported, but I have found this https://github.com/scrapinghub/splash/blob/master/splash/tests/lua_modules/emulation.lua script which mimics the actual python code that runs render.html (and other endpoints).

This (should) allow mimicing (as much as possible) the python code that runs render.html and should be easy to extend with lua/js code.

Note: you want to add the following lines at the bottom of emulation.lua:

function main(splash)
  return {
    html = emulation.render_html(splash)
  }
end

Important: you must add --disable-lua-sandbox to splash (docker) command line