--- tags: - generated_from_trainer datasets: - Graphcore/vqa-lxmert metrics: - accuracy model-index: - name: vqa results: - task: name: Question Answering type: question-answering dataset: name: Graphcore/vqa-lxmert type: Graphcore/vqa-lxmert args: vqa metrics: - name: Accuracy type: accuracy value: 0.7242196202278137 --- # vqa This model is a fine-tuned version of [unc-nlp/lxmert-base-uncased](https://huggingface.co/unc-nlp/lxmert-base-uncased) on the [Graphcore/vqa-lxmert](https://huggingface.co/datasets/Graphcore/vqa-lxmert) dataset. It achieves the following results on the evaluation set: - Loss: 0.0009 - Accuracy: 0.7242 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data [Graphcore/vqa-lxmert](https://huggingface.co/datasets/Graphcore/vqa-lxmert) dataset ## Training procedure Trained on 16 Graphcore Mk2 IPUs using [optimum-graphcore](https://github.com/huggingface/optimum-graphcore). Command line: ``` python examples/language-modeling/run_clm.py \ --model_name_or_path gpt2 \ --ipu_config_name Graphcore/gpt2-small-ipu \ --dataset_name wikitext \ --dataset_config_name wikitext-103-raw-v1 \ --do_train \ --do_eval \ --num_train_epochs 10 \ --dataloader_num_workers 64 \ --per_device_train_batch_size 1 \ --per_device_eval_batch_size 1 \ --gradient_accumulation_steps 128 \ --output_dir /tmp/clm_output \ --logging_steps 5 \ --learning_rate 1e-5 \ --lr_scheduler_type linear \ --loss_scaling 16384 \ --weight_decay 0.01 \ --warmup_ratio 0.1 \ --ipu_config_overrides="embedding_serialization_factor=4,optimizer_state_offchip=true,inference_device_iterations=5" \ --dataloader_drop_last \ --pod_type pod16 ``` ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 5e-05 - train_batch_size: 1 - eval_batch_size: 8 - seed: 42 - distributed_type: IPU - total_train_batch_size: 64 - total_eval_batch_size: 128 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 4.0 - training precision: Mixed Precision ### Training results ### Framework versions - Transformers 4.18.0.dev0 - Pytorch 1.10.0+cpu - Datasets 2.0.0 - Tokenizers 0.11.6