Graphcore and Hugging Face are working together to make training of Transformer models on IPUs fast and easy. Learn more about how to take advantage of the power of Graphcore IPUs to train Transformers models at hf.co/hardware/graphcore.
LXMERT model IPU config
This model contains just the IPUConfig files for running the lxmert-base-uncased model on Graphcore IPUs.
This model contains no model weights, only an IPUConfig.
Model description
LXMERT is a transformer model for learning vision-and-language cross-modality representations. It has a Transformer model that has three encoders: object relationship encoder, a language encoder, and a cross-modality encoder. It is pretrained via a combination of masked language modeling, visual-language text alignment, ROI-feature regression, masked visual-attribute modeling, masked visual-object modeling, and visual-question answering objectives. It acheives the state-of-the-art results on VQA anad GQA.
Paper link : LXMERT: Learning Cross-Modality Encoder Representations from Transformers
Usage
from optimum.graphcore import IPUConfig
ipu_config = IPUConfig.from_pretrained("Graphcore/lxmert-base-ipu")