What does pthread_attr_setinheritsched exactly do?

Question

I'm working on a school problem. I don't want the answer to the problem, but I do have a question.

I'm working on POSIX threads, I have 4 tasks, and I need to run task 3 and 4 before task 1 and 2, by using priority.

Before he wen't on his holiday, my teacher gave me a bunch of example code, which I'm trying to figure out. Especially the line pthread_attr_setinheritsched(&tattr1,PTHREAD_EXPLICIT_SCHED);. I can't figure out what it does. I do know that if I delete it, all threads will run without checking their priority. If I don't delete it, the threads won't run at all. So it's a bit finicky to debug.

I provided the full code below (except for the tasks). Can someone explain what that line does precisely?

int main() {
    // everything on core 0
    cpu_set_t mask;
    CPU_ZERO(&mask);
    CPU_SET(0, &mask);
    sched_setaffinity(0, sizeof(mask), &mask);
    
    // attr1
    pthread_attr_t tattr1;
    struct sched_param param1;
    pthread_attr_init(&tattr1);
    pthread_attr_setschedpolicy(&tattr1,SCHED_RR);
    pthread_attr_getschedparam(&tattr1,&param1);
    param1.sched_priority=10;
    pthread_attr_setinheritsched(&tattr1,PTHREAD_EXPLICIT_SCHED); // <---- THIS LINE
    pthread_attr_setschedparam(&tattr1,&param1);
    
    // attr2
    pthread_attr_t tattr2;
    struct sched_param param2;
    pthread_attr_init(&tattr2);
    pthread_attr_setschedpolicy(&tattr2,SCHED_RR);
    pthread_attr_getschedparam(&tattr2,&param2);
    param2.sched_priority=0;
    pthread_attr_setinheritsched(&tattr2,PTHREAD_EXPLICIT_SCHED); // <---- THIS LINE
    pthread_attr_setschedparam(&tattr2,&param2);
    
    // spin up threads
    pthread_t tid_1;
    pthread_create(&tid_1, &tattr1, taskOne, NULL);
    pthread_t tid_2;
    pthread_create(&tid_2, &tattr1, taskTwo, NULL);
    pthread_t tid_3;
    pthread_create(&tid_3, &tattr2, taskThree, NULL);
    pthread_t tid_4;
    pthread_create(&tid_4, &tattr2, taskFour, NULL);
    
    pthread_join(tid_1, NULL);
    pthread_join(tid_2, NULL);
    pthread_join(tid_3, NULL);
    pthread_join(tid_4, NULL);
}

Answer 1

Welcome to the world of OS thread scheduling.

First off, a thread != a thread.

That is to say that this code is not portable; specifically, if you compile this code on one OS that supports POSIX threads, it will not necessarily operate the same on another.

For example, if I run this code on Fedora 35, (assuming that the "tasks" are simple printf statements), then I only get taskOne and taskTwo, where as if I compile this on my macOS (a BSD based kernel), I get all "tasks" output.

This is because thread scheduling (what your code is dealing with) is OS specific, and the POSIX standard even states that this code does not have to produce the same results.

---

With that, I'll answer your question directly instead of going on a diatribe about the various thread scheduling (too which I've asked a couple of questions in the past regarding POSIX thread scheduling).

The pthread_attr_setinheritsched function does as it says; it sets the thread's scheduling inheritance, which basically means that if you create a thread from another thread, you can have that "child" thread "inherit" the "parent" thread's scheduling, or you could be explicit (via PTHREAD_EXPLICIT_SCHED) in setting it's scheduling.

This would be "helpful" if you have a thread that has a certain scheduling policy (say SCHED_FIFO) that you don't want to have to explicitly set for any other threads spawned from that thread, so depending on OS, by calling pthread_create, if you don't set the attributes schedule/priority, the default could be to inherit from the current thread (see the docs for pthread_attr_init).

Again, it should be emphasized that you must understand the OS you're targeting when using the code you have there due to the fact that you are using POSIX thread scheduling (something I'm sure your teacher did not mention, and probably won't).

To your code specifically, the reason why your threads won't run at all is likely due to your sched_get_priority_min does not accept 0 as a priority (again, depending on your OS); if you did something like this:

printf("min: %d\n", sched_get_priority_min(SCHED_RR));
int err = pthread_attr_setschedparam(&tattr2,&param2);
if (err != 0) {
    printf("pthread_attr_setschedparam error: %s\n", strerror(err));
}

You might see that it would output something like the following:

min: 1
pthread_attr_setschedparam: Invalid argument

The Invalid argument is because you're setting a priority of 0, but your OS thread scheduling policy for SCHED_RR does not accept 0 (a lot of Linux kernels do not, but BSD kernels do, which if you compile this same code on something like macOS or OpenBSD, would compile and run "as expected").

---

Again, this is all very OS (and even kernel version) dependant. You can compile the Linux kernel in various ways to handle thread scheduling; so while I know this question is regarding an academic question, it is extremely important to understand the OS kernel you're targeting with code like this.

---

As a complete aside, I know you don't want the answer to your homework, but I will tell you that your professor is likely setting you up for failure unless you're using an RTOS that handles thread scheduling and priority very differently than a typical "desktop" kernel.

The reason I say that is because your code will likely always run in the order you start your threads, no matter what schedule/priority you set (e.g. using SCHED_FIFO instead of SCHED_RR .. see scheduling policy) or your CPU affinity.

Thread priority is not about thread start ordering. That's not how priority works. Priority and scheduling is about which thread should get the CPU resources first via the time-slice scheduling policy of the kernel.

To start the threads in a 3/4-1/2 order, you'd need to employ a semaphore or a few mutex's in order to guarantee thread start ordering.

For example:

.. headers ..

static pthread_mutex_t mtx1 = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t mtx2 = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t mtx3 = PTHREAD_MUTEX_INITIALIZER;
static pthread_mutex_t mtx4 = PTHREAD_MUTEX_INITIALIZER;

void wait_to_start(pthread_mutex_t* mtx) {
    pthread_mutex_lock(mtx);
    pthread_mutex_unlock(mtx);
}

void* taskOne(void* p) { wait_to_start(&mtx1); printf("taskOne\n"); return 0; }
void* taskTwo(void* p) { wait_to_start(&mtx2); printf("taskTwo\n"); return 0; }
void* taskThree(void* p) { wait_to_start(&mtx3); printf("taskThree\n"); return 0; }
void* taskFour(void* p) { wait_to_start(&mtx4); printf("taskFour\n"); return 0; }

int main() {
    ... your other code here ..

    // grab all of the locks first in main thread
    pthread_mutex_lock(&mtx1);
    pthread_mutex_lock(&mtx2);
    pthread_mutex_lock(&mtx3);
    pthread_mutex_lock(&mtx4);

    // spin up threads
    pthread_t tid_1, tid_2, tid_3, tid_4;
    pthread_create(&tid_1, &tattr1, taskOne, NULL);
    pthread_create(&tid_2, &tattr1, taskTwo, NULL);
    pthread_create(&tid_3, &tattr2, taskThree, NULL);
    pthread_create(&tid_4, &tattr2, taskFour, NULL);

    // sequence point/time-slice (helps to ensure all threads actually start)
    usleep(1);

    pthread_mutex_unlock(&mtx3);
    pthread_mutex_unlock(&mtx4);
    
    // sequence point/time-slice (helps to ensure thread 3/4 actually continue)
    usleep(1);

    pthread_mutex_unlock(&mtx1);
    pthread_mutex_unlock(&mtx2);

    ... your final code here ...
}

I hope that can help you understand your code a little more.