I am new to ML and programming for ML. I am trying to do a grid search on the parrot-paraphraser_for_t5 transformer from hugging face. There are two issues I am facing:
- I am not sure of the training_data format.
- I am not sure about how to mention the parameters for getting the best combination.
This is how I am currently specifying my trainer:
trainer = Trainer(
model=model,
args=training_args,
train_dataset = [
{"input_text": "Paraphrase: 'The quick brown fox jumps over the lazy dog.'", "target_text": "'A speedy brown fox leaps over the lethargic canine.'"},
{"input_text": "Paraphrase: 'I enjoy reading books.'", "target_text": "'Reading books is something I find pleasurable.'"},
{"input_text": "Paraphrase: 'The weather is nice today.'", "target_text": "'Today's weather is pleasant.'"},
],
data_collator = transformers.DataCollatorForSeq2Seq(tokenizer=tokenizer, padding=True)
)
and below are my training arguments, placeholders for now.
training_args = TrainingArguments(
output_dir="./results",
per_device_train_batch_size=params["per_device_train_batch_size"],
num_train_epochs=params["num_train_epochs"],
learning_rate=params["learning_rate"],
)
Huggingface website mentions that Parrot offers knobs to control Adequacy, Fluency and Diversity. Are these the hyper parameters I should be looking at? If so, how can I specify them in my code?
I am using T5Tokenizer and DataCollatorForSeq2Seq as my collator.
Please find the link to the hugging face transformer here and the link to my google colab code here.
Thanks in advance!