I want to call a function with dlsym and it is a member function of an object, I have the pointer, but it is not working. The function lives in the main executable, and I am loading it from a shared library. The function is not exported so direct dlsym fails, using gdb break I calculated the offset of this function and other known exported function, so I do dlsym in the exported function and calculate the offset to the other function. I need to call it but I can't pass the args correctly, the first argument should be the "this" implicit pointer since it is a nonstatic member function.
The function definition is this:
_int64 __fastcall gplayer_controller::DebugCommandHandler(gplayer_controller *const this, int cmd_type, const void *buf, size_t size)
my code is this:
/* open the needed object */
void *handle = dlopen(NULL, RTLD_LOCAL | RTLD_LAZY);
if(handle == NULL){
printf("error w/ dlopen\n" );
}
int (*fptr)(controller *, int, mma *, size_t);
fptr = (int (*)(controller *, int, mma *, size_t))dlsym(handle, "lua_pushboolean");
if(fptr == NULL){
printf("error w/ funcion\n" );
}
else{
printf("found, ptr: %p\n", fptr);
}
gobject_imp *pImp = (gobject_imp*)skill->GetPlayer()->GetObject().GetImpl();
int (*fptr2)(controller *, int, mma *, size_t) = fptr - 5638326;
printf("ptr calculation...: %p\n", fptr2);
mma _mma;
_mma.cmd = 2040;
_mma.skillid = 15000;
_mma.level = skill->GetLevel() + 1;
printf("data controller %p\n",pImp->_commander );
(*fptr2)(pImp->_commander,2040,&_mma,10);
the first parameter is the "this" pointer, the other 3 are the normal function params
I finally got it working, now I can call that member function like if it where a normal one.
The problem with my code, was that the pointer I was getting was wrong, with gdb I got another address with command info frame, but in fact that pointer was also wrong, the function was been called but with all params in 0, after checking in IDA pro , the address gdb was giving me was after the function started, right after the function setted input params.
GDB was giving me the address 0000000000562875, right after moving registers with params to rbp.
I recalculated my ptr to the first line of the function (562847) in IDA. And it worked!! I can call the function, the first param I sent as the implicit "this" param works like a charm.