Why is this while loop not reading the input correctly? Keeps printing out p

Question

I was using scanf previously to get input for this but I kept having problems and after a bit of research it definitely looks like fgets is a safer option when asking for input.

As they can technically input anything they want, but I want to only accept Y or N lower and upper case.

However the fgets in the while loop for some reason isn't working, no matter what I enter into that fgets it will always print out P and it will never exit the while loop.

I'm just wondering what the issue there is. Because in this program, as long as its Y or N it shouldn't even go to the while loop.

But it always goes to the while loop and it never really inputs the right thing it seems

int main() {
char *answer = malloc (MAX_NAME_SZ);
printf("Welcome to our Physics App!");

printf ("\nWould you like to begin? [Y/N]");
fgets (answer, MAX_NAME_SZ, stdin);

if(*answer == 'y' || *answer == 'Y'){
    printf("\nprogram starting now...");
}
else if(*answer == 'n' || *answer == 'N'){
    printf("\nThank you, program will close now....");
    return 0;
}
while(*answer !='n' || *answer != 'N' || *answer != 'y' || *answer != 'Y'){
    printf("\nPlease enter [Y], or [N]");
    fgets (answer, MAX_NAME_SZ, stdin);
    printf("Your answer was %c", answer);
}