FreeRTOS mutex priority inheritance problem if changing priority of task

Question

Here is the scenario I am not sure will be the problem.

Foo()
{
TakeMutex()
//some critical code
GiveMutex()
}

Task A    priority 5
Task B    priority 1

TaskB{ Foo() }
TaskA{ Foo() }

now in some other task, it may change the priorities of Task A and B. Lets say that Task B calls Foo and takes the mutex. Now while B has the mutex, Task A calls foo and attempts to take the mutex. Because of priority inheritance for mutex, Task B will now become the priority of Task A which is 5.

Task A    priority 5
Task B    priority 5 inherited

Now at this moment, some other task attempts to change the priority of Task
B to 8 using vTaskPrioritySet(). 

Task A    priority 5
Task B    priority 8 the set value if even? or does it stay 5 returning 8? 

The question is, after Task B lets go of the mutex, what priority would it return to? would it go back to its original priority 1 or keep its set value. What about if instead the scenario was that Task A was changed to a lower or higher priority value. Is there any permutation of this scenario that would cause unexpected behavior?

Answer 1

Most of the RTOS implementations have not complete mutex priority inheritance implementation. E.g. quoting from FreeRTOS - Priority inheritance proposal

The FreeRTOS priority inheritance mechanism is described as “simplified”,

Answer 2

The goal of priority inheritance in freeRTOS is to minimize the effects of priority inversions, not to solve them completely. It's a balance between implementation complexity, performance penalties, etc.

Answering to your question "The question is, after Task B lets go of the mutex, what priority would it return to?", it will return to its original priority after releasing the mutex. Pre-emption can, and most likely, happen during this operation.