how can we get the prediction logits in the lxmert model using hugging face library? It's fairly easy to get in visualbert, but I'm not able to get it with the lxmert model. In case of visualbert model, the keys I'm getting are :
['prediction_logits', 'seq_relationship_logits', 'attentions']
and with the help of lxmert mode, the keys are :
['language_output', 'vision_output', 'pooled_output', 'language_attentions', 'vision_attentions', 'cross_encoder_attentions']
Even though there's a mention of prediction logits in the documentation I am not able to get them, if someone can help that would be great.
EDIT : Link to colab notebook for lxmert.
Use LxmertForPreTraining instead of LxmertModel:
Output:
P.S.: You can control the pertaining task heads via the configuration fields
task_matched
,task_mask_lm
,task_obj_predict
, andtask_qa
. I assume you are only interested inmask_lm
following your comment. That means you should initialize your model as follows: