How to parameter HuggingFace for multi CPU training?

Question

I've follow some of the post I found online by setting the .to('cpu') method

model = AutoModelForMaskedLM.from_pretrained(MODEL_TYPE).to('cpu')

Then in the training argument: I've set the number of device to 8 (total CPU on the device) and set the no_cuda=True

training_args = TrainingArguments(
    output_dir=output_dir,
    overwrite_output_dir=True,
    evaluation_strategy="steps",
    learning_rate=2e-5,
    weight_decay=0.01,
    logging_steps = 10,
    save_total_limit=5,
    load_best_model_at_end=True,
    gradient_accumulation_steps=2,
    per_device_train_batch_size=8,
    prediction_loss_only=True,
    remove_unused_columns=False,
    no_cuda=True
)

The code starts executing and the HF progress bar appears as expected.

But when I check the task manager CPU usage hover around 50%, while using Chrome in the meantime, so it's clearly not taking advantage of all CPU available but I don't know what I'm missing.