PyTorch: AttributeError: 'torch.dtype' object has no attribute 'itemsize'

Question

I am trying to follow this article on medium Article.

I had a few problems with it so the remain chang eI did was to the TrainingArguments object I added gradient_checkpointing_kwargs={'use_reentrant':False},.

So now I have the following objects:

peft_training_args = TrainingArguments(
    output_dir = output_dir,
    warmup_steps=1,
    per_device_train_batch_size=1,
    gradient_accumulation_steps=4,
    max_steps=100, #1000
    learning_rate=2e-4,
    optim="paged_adamw_8bit",
    logging_steps=25,
    logging_dir="./logs",
    save_strategy="steps",
    save_steps=25,
    evaluation_strategy="steps",
    eval_steps=25,
    do_eval=True,
    gradient_checkpointing=True,
    gradient_checkpointing_kwargs={'use_reentrant':False},
    report_to="none",
    overwrite_output_dir = 'True',
    group_by_length=True,
)

peft_model.config.use_cache = False

peft_trainer = transformers.Trainer(
    model=peft_model,
    train_dataset=train_dataset,
    eval_dataset=eval_dataset,
    args=peft_training_args,
    data_collator=transformers.DataCollatorForLanguageModeling(tokenizer, mlm=False),
)

And when I call peft_trainer.train() I get the following error:

AttributeError: 'torch.dtype' object has no attribute 'itemsize'

I'm using Databricks, and my pytorch version is 2.0.1+cu118

Answer 1

if you use transformers version latest(v4.39.1), plz try to downgrade to v4.38.2. I solved it that way.

Answer 2

The itemsize attribute doesn't seem to have documentation in PyTorch 2.0.x, which suggests that it doesn't exist in that version. (Try searching "itemsize" in the docs for versions 2.0 and version 2.2 yourself.) The probable reason for this is that one of the packages you're using assumes a Pytorch version greater than the one you're using.

Moreover, if you look at the Medium article author's own notebook for this tutorial here, you will see that pip install errors out, suggesting their code isn't runnable.

The best solution I can propose (given that the medium author didn't seem to provide a list of requirement versions) is to create a conda environment which will try to resolve the various package dependencies for you, or to reach out to the author for a precise requirements file so that you can replicate their work. If they cannot do this for you, then don't trust their results.

Answer 3

Upgrading to torch-2.1.2 will resolve that error.