I've started learning about information security and trying to solve some tasks about this topic. The program i have to exploit contains stack canary. It's a simple program asking password for 3 times and if the password matches it will print out hidden information. The program never interrupts until asking all 3 times the question and only after 3d time it will show error, SIGSEGV or Stack Smashed detected. I don't know the code of the program, only binary file. The program uses such functions as strcmp, printf, strlen, puts, stack_chk_fail in main and open, return_hidden_file, write, read, stack_chk_fail for return_hidden_file function. I was told this program doesn't use rbp and i dont need to use it for exploiting in this case. There is buffer for entering password exists, it's like buffer[10] and one more buffer in return_hidden_file [0x32]. There is also one more function called upload_password. It has fopen, upload_password + 61, puts, exit, fread, fclose in it. I've seen other topics about this in SO but it didn't help.
I'm expecting to make the program to print out the hidden word. Firstly, i thought i have to BOF the program with value of whole buffer(let's say it's 0x48) minus 8 bytes for canary and minus 8 empty bytes following the canary. Because i thought the concept was to BOF with 0x48-8-8 in the first time question about password, then to get leaked stack canary address and bypass that place, then input some more values for the second time question and get the hidden word at the 3d try. But this didn't word, and many other things didn't work too, including methods i saw in the internet. So i'm completely lost and confused. Which moment i can exploit the program? When it's asking the question about password for the first time, second, or third? Do i need to know the address of stack canary and if not, then how do i bypass it? Do i need to BOF on the first time question? Do i need to use return address of the hidden_file function in exploitation code? May be need to exploit BOF in heap memory? help me with something:(