How to out put attentions in Transformers BART model

36 Views Asked by At

New to Transformers. When testing BART, I try to output attentions, so set the parameter Output_attentions=True, but results is missing the attention tensor.

Code:

from transformers import AutoTokenizer, BartForConditionalGeneration

model = BartForConditionalGeneration.from_pretrained("facebook/bart-large-cnn",output_attentions=True)
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn")

ARTICLE_TO_SUMMARIZE = (
    "PG&E stated it scheduled the blackouts in response to forecasts for high winds "
    "amid dry conditions. The aim is to reduce the risk of wildfires. Nearly 800 thousand customers were "
    "scheduled to be affected by the shutoffs which were expected to last through at least midday tomorrow."
)
inputs = tokenizer([ARTICLE_TO_SUMMARIZE], max_length=1024, return_tensors="pt")

# Generate Summary
summary_ids = model.generate(inputs["input_ids"], num_beams=2, min_length=0, max_length=20)
tokenizer.batch_decode(summary_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

Output:

summary_ids:
tensor([[   2,    0, 8332,  947,  717, 1768,    5,  909, 4518,   11, 1263,    7,
         5876,   13,  239, 2372, 2876, 3841, 1274,    2]])

Any help is appreciated.

0

There are 0 best solutions below