GPT-4V responses only ✓ ✓ ✓
LLaVA responses only ✓ ✓ ✓
Table 4: Controlled settings for multimodal preference dataset exploration. We decompose the
preference datasets into prompts, chosen and rejected responses and we then aim at identifying factors
contributing to the dataset quality.
responses (e.g., style change between MLLMs) and permit a tighter control on ablations, we replace
rejected responses from the original VLFeedback samples with corrupted versions of the preferred
responses. Similar to the method in (Zhou et al., 2024), we leverage GPT-4 to specifically introduce
realistic hallucinations, assisted by a few shots for illustration (see Appendix C.1).
|Dataset|POPE ↑ MMHAL ↑ MMHALv ↑ LLaVAW ↑ VQAT ↑ GQA ↑ MMVet ↑ Recallcoco ↑|
|---|---|
|Baseline
VLFeedback (5k)
+ corrupting strategy|86.40
2.95
2.75
80.85
64.85
64.23
43.94
68.13
86.31
2.92
3.00
83.10
65.06
64.09
43.21
68.03
85.59
3.39
3.33
86.65
65.20
63.87
37.98
68.66|


LLaVA prompts 87.63 2.85 2.96 86.55 65.13 64.25 41.47 70.44

Understanding Alignment in Multimodal LLMs: A Comprehensive Study

Elmira Amirloo
Jean-Philippe Fauconnier
Christoph Roesmann
Christian Kerl
Rinu Boney
Yusu Qian
Zirui Wang
Afshin Dehghan
Yinfei Yang
Zhe Gan
Peter Grasch


GPT-4V responses only 86.78 3.30 3.02 86.77 65.06 64.02 40.14 69.08
LLaVA responses only 87.52 3.03 3.01 88.64 65.30 64.19 42.16 70.13
Table 5: Dataset ablations. We started from the public VLFeedback dataset with its diverse prompts
and responses, and we then applied targeted sampling and corruption to isolate the factors contributing
to the quality of a preference dataset.
**Results** Following Section 5.2, we apply DPO alignment on the LLaVA 1.6-7B model, and we limit
all the datasets to 5,000 samples. In Table 5, we report the results of this experiment. First, we show
that our corruption strategy achieves improvements over the baseline comparable in magnitude to
the ranking-based preference signal in the original VLFeedback data. In some benchmarks, like
MMHAL-Bench-V, we even observe improvements, while notably MMVet shows some regressions.
Nevertheless, we argue that this represents a reasonable baseline to adopt for easier iteration on the
following ablations. In Appendix C.2, we provide more analysis on this strategy.
Next, we explore the impact of novelty of the prompts used for alignment, by sampling another
5k preference data generated with the same corruption mechanism solely from prompts that are a
part of the LLaVA SFT mixture. These are examples that the base model would have already been
trained on during the SFT stage. Interestingly, it appears that using novel prompts does not offer
substantial benefits. We still observe comparable lift on LLaVABench-in-the-Wild, and while
MMHAL-Bench-V shows less dramatic improvement over the baseline compared to the more diverse
corruption-based sample, this may be due to more verbose responses, as indicated by higher recall.
POPE even improves somewhat significantly and the regression in MMVet is also less pronounced.
Finally, we explore the impact of the construction of the accepted response in the alignment data. One
could argue that for responses derived from stronger model such as GPT-4V, improvements may also
be the result of learning from this stronger teacher model. Therefore, we conduct two experiments:
one, where we sample data where the preferred response comes from GPT-4V only, and one where
the preferred response comes from LLaVA 1.5-7B, a model generally weaker than the base model
under alignment in this experiment. Interestingly, we do not observe any benefit from learning from
GPT-4V generated responses, in fact, our results suggest that positive samples derived from LLaVA
1.5-7B led to a slightly stronger model post alignment.
These findings suggests that useful preference data can be derived cheaply, even from responses
from relatively weaker models, as long as one can effectively sample and identify relatively desirable
answers from the model as their preferred response [6], and introduce targeted corruption to create
dispreferred responses.
In the following Section 5.4, we will discuss how one can avoid both the need for sampling preferred
model responses as well as the need for an external model to introduce corruption with BDHS.
5.4 ABLATIONS ON BDHS
Section 3.4 introduces BDHS as a technique to generate corrupted responses directly using the model
subject to alignment. While our proposed approach is purely based on image attention masking,
we also evaluate a variant that consumes noisy images instead, motivated by the teacher-forced
_POVID-style image distortion_ introduced in Zhou et al. (2024) (see Section 3.3). In the following,
BDHS with attention masking ( _ρ_ th = 0 _._ 99 and _N_ BDHS = 5) is denoted as BDHSattn and BDHS with
noisy images in the input as BDHSnoise. The number of additive noise steps for BDHSnoise is set to
_N_ = 500 similar to the image distortion in Zhou et al. (2024).
All ablations in Table 6 are based on our 5k subset of POVID as introduced in Section 5.2. As
described in Section 3.2, POVID contains LLaVA Instruct responses _y_ [+] as well as GPT-4V corrupted
non-preferred responses _y_ _[−]_ . While _y_ [+] is shared between all ablations, we start with substituting _y_ _[−]_ from external supervision by the BDHS model response ˜ _y_ _[−]_ and invoke standard DPO as
shown in the first 3 rows after the LLaVA 1.6-7B baseline results. The proposed variants consistently improve over the baseline for POPE and LLaVABench-in-the-Wild. They regress
on MMHALBench, however, as discussed in Section 4.2, this benchmark has limitations so we
mainly focus on MMHALBench-V instead for which all BDHSattn variants perform comparable to
the baseline while the online rollout of ˜ _y_ _[−]_ even improves over it. Notably, we also observe significantly higher Recall [coco], suggesting richer responses. BDHSnoise results in lower scores for
LLaVABench-in-the-Wild while the attention masking approach BDHSattn almost maintains
the baseline scores.
|y−from external supervision y˜−derived from policy|POPE↑ MMHAL↑ MMHALV ↑ LLaVAW↑ VQAT ↑ GQA↑ MMVet↑ Recallcoco ↑|
|---|---|
|–
–|86.40
**2.95**
2.75
80.85
64.85
64.23
43.94
68.13|
|–
BDHSnoise (Offine, ours)
–
BDHSattn (Offine, ours)
–
BDHSattn (Online, ours)|88.60
2.37
2.48
84.53
65.05
64.14
41.38
75.16
88.75
2.61
2.71
**86.33**
65.07
63.97
43.4
**75.58**
**88.83**
2.80
**2.99**
85.03
65.09
63.65
43.12
74.09|
|GPT-4V (POVID)
–
GPT-4V (POVID)
POVID-style image distortion
GPT-4V (POVID)
BDHSnoise (Offine, ours)
GPT-4V (POVID)
BDHSattn (Offine, ours)
GPT-4V (POVID)
BDHSattn (Online, ours)|88.18
2.93
2.93
81.89
64.90
**64.34**
43.39
71.80
88.33
2.84
2.64
80.15
64.21
63.79
41.28
69.39
88.58
2.76
2.45
84.36
65.31
64.26
43.95
75.05
88.56
2.85
2.72
85.35
65.39
64.11
43.26
75.05
88.38
2.82
2.81
84.01
**65.42**
64.30
**45.46**
74.00|
Table 6: Ablation results for BDHS including baseline and reference approaches. All results based
on LLaVA 1.6-7B, using DPO and the POVID (5k) sample for the source of images and prompt.
Whenever both _y_ _[−]_ from external supervision and ˜ _y_ _[−]_ derived from policy (either online or offline)
are incorporated, the average loss is computed using (5).
The lower partition of Table 6 starts with plain DPO on the POVID (5k) dataset as reference and
then each subsequent approach incorporates both the existing response _y_ _[−]_ from external supervision
as well as ˜ _y_ _[−]_ derived from the policy. Hereby, the two non-preferred responses are incorporated
into the DAP framework by averaging the losses of ( _y_ [+] _, y_ _[−]_ ) and ( _y_ [+] _,_ ˜ _y_ _[−]_ ) according to Equation (5).
Therefore, the Online-BDHS method uses Online-DPO in a considerable simplified setting compared
to the full Online-DPO realization (4), as the formulation presented here does not depend on a
dedicated external annotator (see Section 2.3).
All the BDHS ablations improve significantly on LLaVABench-in-the-Wild compared to the
DPO baseline and POVID-style image distortion. The BDHSattn with attention masking performs
6In this ablation, preferred responses were selected from model responses that were ranked as the best among
the sampled model responses per example in VLFeedback. In this setting the chosen response can be assumed
to be of reasonable quality as it was chosen to be at least better than other models’. While this still indirectly
exploits the ranking in the VLFeedback data, all that is required is a way to sample reasonable model answers,
which is generally much more readily available in practical scenarios than paired preference data, for example
via cheap user feedback (thumbs up / down).
Held-Out Eval Dataset
Base Model Train Dataset
POVID RLHF-V VLFeedback
LLaVA 1.5-7B POVID 0.99 0.24 0.56
LLaVA 1.5-7B RLHF-V 0.12 0.86 0.52
LLaVA 1.5-7B POVID + RLHF-V 0.98 0.76 0.53
LLaVA 1.5-7B VLFeedback 0.61 0.54 0.81
LLaVA 1.6-7B POVID 0.99 0.34 0.59
LLaVA 1.6-7B POVID + RLHF-V 0.97 0.68 0.63
LLaVA 1.6-7B VLFeedback 0.76 0.53 0.82
Table 7: Reward model accuracy on the held-out validation set.
significantly better on MMVet compared to BDHSnoise. Notably, BDHSattn consistently outperforms
the POVID-style image distortion across all benchmarks. We follow the published implementation of
Zhou et al. (2024), however, surprisingly the _POVID-style image distortion_ performs worse compared
to plain DPO via POVID (5k), which differs from the LLaVA 1.5-7B alignment results in their paper.
Presumably, the non-sensical responses from teacher-forcing could lower the performance while
trading off with the existing GPT4-V preference pairs.
While online approaches with BDHS improve on certain benchmarks, we emphasize that even the
offline dataset created with BDHSattn and without additional response from external supervision
already constitutes a cost-effective baseline that consistently performs well across all benchmarks.
Unless otherwise stated, BDHS in the following sections generally refers to BDHSattn.
5.5 RL-BASED ALIGNMENT
To evaluate RL-based alignment methods we followed the established recipe of training a reward
model on a preference dataset and then using an RL algorithm to optimize the MLLM to maximize
the reward of responses sampled from the policy. We chose PPO and RLOO due to their popularity
in the LLM literature.
**Reward Model Training and Evaluation** We analyze the utility of datasets available in the
community for reward model training by training on POVID, RLHF-V and VLFeedback preference
datasets. To evaluate such created reward models in isolation, we hold out a small validation set split
from the original dataset and report classification accuracy of the trained reward model, i.e. its ability
to differentiate the chosen from the rejected response in POVID, RLHF-V, and VLFeedback sets.
These held out validation sets are not used for reward model training.
Table 7 shows the performance of the reward models trained on different datasets across all validation
sets. The model trained on VLFeedback shows the best generalization across the different datasets,
as may be expected given its significantly larger size and higher diversity. In contrast, reward models
trained on POVID and RLHF-V show notably poor generalization to their respective counterpart,
while achieving high scores on their own held out portions. We hypothesize that the reward model
may learn to recognize and prefer the respective (original) policy response before corruption (POVID)
or enhancement (RLHF-V), which could explain the performance being significantly below a random
choice baseline. To strengthen our hypothesis, we also combine the POVID and RLHF-V sets for
reward model training and observe that both LLaVA 1.5-7B and LLaVA 1.6-7B are able to learn a
more balanced objective, although even for such a combined training set we still observe limited
generalization to VLFeedback.
**RL Training and Evaluation** We used the POVID and VLFeedback based reward models for PPO
and RLOO training. Table 8 shows the scores of the best models trained via PPO and RLOO.
Mirroring the observed lack in generalization in our reward model experiments, we found that using
POVID-based reward model resulted in collapse of responses during the RL training. Only the
use of the reward model trained on the much larger VLFeedback dataset allowed for stable RL
training without model collapse. We hypothesize that besides the larger size, VLFeedback may
be more aligned with the downstream objective of the reward model due to its construction by
ranking sampled model responses, compared to POVID, which aims to produce minimally different
Alignment Dataset _RM_ Dataset _P_ POPE _↑_ MMHAL _↑_ MMHAL _[V]_ _↑_ LLaVA [W] _↑_ VQA _[T]_ _↑_ GQA _↑_ MMVet _↑_ Recall [coco] _↑_
Baseline  -  - 86.40 2.95 2.75 80.85 64.85 64.23 43.94 68.13
DPO - POVID 88.09 3.16 3.07 78.05 64.56 64.12 40.60 73.48
PPO POVID POVID
Policy training not stable
RLOO POVID POVID
PPO VLFeedback POVID 87.54 3.02 3.09 80.17 63.90 64.04 40.51 67.19
RLOO VLFeedback POVID 87.17 2.94 2.72 78.72 63.59 63.72 42.25 64.57
Table 8: RL-based alignment of LLaVA 1.6-7B, DPO baseline included for reference. RL-based
alignment methods use a reward model based on LLaVA 1.6-7B, Dataset _RM_ refers to the dataset
used to train the reward model, Dataset _P_ to the set of images and prompts used for RL alignment.
preference pairs. Nevertheless, even the stronger VLFeedback-based reward model did not allow us
to reliably outperform a much simpler DPO baseline [7] .
These observations indicate that reward model training with subsequent RL alignment could perhaps
require more carefully curated data, e.g., with more focus on diversity, than direct alignment methods
where both POVID and VLFeedback individually achieve strong improvements. In addition to
inherently stronger reward models, perhaps basing them on more powerful base models, it also
suggests that the approach introduced in the concurrent work of Yu et al. (2024), which introduces a
symbolic reward formulation based on scores from a VQA model verifying statements made by the
policy may be a promising avenue for future research.
Another interesting observation is that the RL aligned models show similar evaluation trends as the
DAP aligned models, where both use POVID prompts and images for the training of the policy. For
example, compared to the base model they show some improvement in POPE, and MMHalBench
(both variants), with some regressions in LLaVABench-in-the-Wild, TextVQA, GQA, and
MMVet. These trends are distinct to what is seen when using direct preference alignment on
VLFeedback data as shown in Table 3. This is remarkable as the RL aligned models do of course not
use the chosen and rejected responses present in the POVID dataset, instead getting their feedback
signal entirely from the reward model which is trained on VLFeedback data. We observe a similar
trend in Section 5.6, where in a purely online setting, the choice of input prompts and images
significantly impacts alignment results.
5.6 ONLINE-DPO & MIXED-DPO
We apply Online-DPO and Mixed-DPO to both the POVID and the RLHF-V dataset. The results are
summarized in Table 9. Consistent with our observations on the POVID dataset, applying Mixed-DPO
- which combines elements of DPO and Online-DPO – typically results in a moderating effect on
performance outcomes. The results often span a range slightly broader than the highest and lowest
performances achieved by DPO and Online-DPO. This variability is attributed to the probabilistic
nature of the online sampling in Online-DPO.
On the RLHF-V dataset, where Online-DPO consistently outperforms DPO across all benchmarks,
the moderating effect of Mixed-DPO proves not beneficial, as the offline DPO component contributes
minimally to the overall model performance. Nevertheless, Mixed-DPO remains a valuable strategy in
scenarios where, as observed in the experiments on the POVID dataset, offline and Online-DPO show
complementary improvements, leveraging the strengths of both to optimize overall performance.
5.6.1 HOW DOES A STRONGER ANNOTATOR AFFECT THE PERFORMANCE OF ALIGNED MODEL?
**Annotator Evaluation** We used LLaVA 1.6-34B as the annotator. To verify its capability to accurately
judge different responses, we evaluated it on the sample held-out part of three datasets we used for
evaluating the reward model in Section 5.5. Results are summarized in Table 10. For further details
on the prompts used and qualitative examples of the annotator’s outputs, please refer to Appendix D.
Table 11 presents a comparison of the effects of Online-DPO with two different annotators.
7We also found that models achieving higher reward during RL training, did not perform better than models
with lower reward and less KL divergence, i.e., models with higher _β_ parameter performed better on the
benchmarks. None of the RL algorithms clearly outperformed the others.
Alignment Dataset POPE _↑_ MMHAL _↑_ MMHAL _[V]_ _↑_ LLaVA [W] _↑_ VQA _[T]_ _↑_ GQA _↑_ MMVet _↑_ Recall [coco] _↑_
   -   - 86.41 3.06 2.71 78.96 64.22 64.22 43.94 68.13
DPO POVID 88.09 3.16 3.07 78.63 64.56 64.12 40.60 73.48
Online-DPO POVID 86.49 2.88 2.94 82.61 64.88 64.31 43.26 68.45
Mixed-DPO POVID 88.03 2.83 3.10 82.75 64.93 64.47 42.80 74.53
DPO RLHF-V 83.86 3.15 3.26 70.58 64.75 62.89 37.16 64.26
Online-DPO RLHF-V 85.40 3.10 3.27 79.66 64.94 64.05 41.01 68.13
Mixed-DPO RLHF-V 85.57 2.94 3.16 78.46 65.06 64.10 41.10 67.82
Table 9: The effect of Mixed-DPO, using LLaVA 1.6-7B as the base model.
While using Online-DPO with LLaVA 1.67B as the judge can enhance the overall performance of the model, the stronger annotator
seems to provide more consistent improvements
across various benchmarks.
VLFeedback (eval) 79.10 90.91
RLHF-V (eval) 81.88 93.90
POVID (eval) 92.96 98.55
Table 10: Performance of the annotators on different
preference datasets.
Dataset LLaVA 1.6-7B LLaVA 1.6-34B
Concurrent to us, Yu et al. (2024) proposed an
annotation approach that segments the annotation process into easier sub-tasks, with each task being individually scored. These scores are then
aggregated to form an overall score that rates the responses. This method can potentially enable
weaker models to still provide strong supervision signals during the alignment process. Moreover,
exploring the use of stronger base models and diverse datasets, both in terms of size and variety,
could further enhance the effectiveness of the online approach. We leave the detailed investigation of
these aspects for future work.
Model Dataset Annotator POPE _↑_ MMHAL _↑_ MMHAL _[V]_ _↑_ LLaVA [W] _↑_ VQA _[T]_ _↑_ GQA _↑_ MMVet _↑_ Recall [coco] _↑_
LLaVA 1.6-7B - - 86.40 2.95 2.75 80.85 64.85 64.23 43.94 68.13
LLaVA 1.6-7B POVID LLaVA 1.6-7B 86.54 2.52 2.72 81.55 64.93 64.18 40.73 67.40
LLaVA 1.6-7B POVID LLaVA 1.6-34B 86.49 2.88 2.94 82.61 64.88 64.31 43.26 68.45
Table 11: Comparison of Online-DPO with a strong annotator (i.e., LLaVA 1.6-34B) and a weak
annotator (i.e., LLaVA 1.6-7B).
5.7 COMPARISON OF DIFFERENT OFFLINE ALIGNMENT METHODS
While we conducted most of our experiments using DPO for comparability with other works in the
community, we also ran a few experiments to investigate whether other popular offline methods could
improve the results. Results are summarized in Table 12.
Our results indicate that both IPO and SLiC, similar to DPO, boost the model’s performance across
most hallucination benchmarks. Additionally, these methods demonstrate improvements in more
open question-answering benchmarks. We anticipate that Online-IPO and Online-SLiC will yield
enhancements over their offline counterparts — similar to the improvements observed with OnlineDPO over DPO — as examined in Guo et al. (2024). However, this study is beyond the scope of
this paper and is left for future work. Primarily, we aim to highlight the importance of considering
different alignment objectives, emphasizing that the choice between offline objectives in different
setups can impact the effect of the alignment pipeline.
Alignment Dataset POPE _↑_ MMHAL _↑_ MMHAL _[V]_ _↑_ LLaVA [W] _↑_ VQA _[T]_ _↑_ GQA _↑_ MMVet _↑_ Recall [coco] _↑_
   -   - 86.40 2.95 2.75 80.85 64.85 64.23 43.94 68.13
DPO POVID 88.09 3.16 3.07 78.63 64.56 64.12 40.60 73.48
IPO POVID 87.62 3.11 3.11 82.34 65.09 64.47 43.99 69.81
SliC POVID 88.28 3.17 3.15 81.99 64.59 64.11 41.51 74.32
Table 12: Comparison of different offline alignment methods based on LLaVA 1.6-7B.
6 CONCLUSION AND FUTURE WORK
In this study, we explore the role of preference alignment in enhancing the performance of MLLMs,
with a particular focus on reducing hallucinations. A commonly proposed explanation for hallucinations in MLLMs is their tendency to overlook image content and instead rely on inherent language
biases. To address this, we assess various alignment strategies across different datasets and alignment
methods. We categorize alignment algorithms into offline and online strategies and demonstrate
that a hybrid approach can offer benefits in specific scenarios. We also do a thorough study on the
existing multimodal preference datasets, identifying strengths and weaknesses associated with each,
and providing insights into how certain types of preference data can enhance model performance.
Leveraging these insights, we develop our own preference dataset and introduce a novel data sampling
strategy, BDHS. When applied to the LLaVA 1.6 model, these methods lead to notable improvements
across various benchmarks, confirming the potential of tailored preference alignment strategies in
refining the capabilities of MLLMs. A significant advantage of this approach is its ability to operate
effectively using only SFT data, eliminating the need for a superior model, human labelers, or other
complex means of constructing preference data.
This study not only enhances our understanding of preference alignment but also establishes a
foundation for further research into MLLM preference alignment. Specifically, we identify several
gaps in the community’s approach to aligning MLLMs:
 - While considerable research has been conducted on various alignment methods, including both
online and offline approaches, for LLMs, these studies are less common in the context of MLLMs.
For instance, RLH(AI)F is extensively discussed in LLM literature, highlighting its potential over
the more simple methods like DPO (Ahmadian et al., 2024; Xu et al., 2024). We have provided
some insights into RL-based alignment for MLLMs and the evaluation of reward models, yet we
believe there remains a significant gap between LLM and MLLM research in this domain.
 - A better hallucination benchmark can help our understanding of model improvements. We discuss
some of the shortcomings of current hallucination benchmarks in Sections B.1, B.2 and B.3.
However, the development of an effective hallucination benchmark remains an active area of
research.
 - We thoroughly analyze various aspects of published multimodal preference data. However, the
coverage of these datasets is still insufficiently studied. The lack of comprehensive coverage in
these datasets may contribute to the absence of significant improvements in some benchmarks.
This paper has highlighted key advancements and existing challenges in preference alignment for
MLLMs. Our findings point out important gaps that need addressing. We hope these insights inspires
further research and helps the community tackle ongoing challenges in this field.
ACKNOWLEDGMENTS
The authors would like to thank Sebastian Brechtel, Meng Cao, Philipp Dufter, Jiaming Hu, Lukas
Jendele, Juan Lao Tebar, Shuang Ma, Dhruti Shah, Will Song, Juergen Wiest, and Haotian Zhang
for their feedback and guidance throughout this project. We would also like to thank the authors of
Zhou et al. (2024), Li et al. (2023a), and Yu et al. (2023b) for their prompt responses and effective
communication with us during the process.
REFERENCES
Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Ahmet Üstün, and
Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human
feedback in llms. arXiv preprint
Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bilal Piot, Remi Munos, Mark Rowland, Michal
Valko, and Daniele Calandriello. A general theoretical paradigm to understand learning from human
preferences. In International Conference on Artifcial Intelligence and Statistics, pp. 4447–4455.
PMLR, 2024.
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou,
and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization,
text reading, and beyond. 2023.
Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. Dress: Instructing large
vision-language models to align and interact with humans via natural language feedback. arXiv
preprint
Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep
reinforcement learning from human preferences. Advances in neural information processing
systems, 30, 2017.
Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao.
Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint
Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, James Zou, Kai-Wei Chang, and Wei Wang.
Enhancing large vision language models with self-training on image comprehension. arXiv preprint
Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In
International Conference on Machine Learning, pp. 10835–10866. PMLR, 2023a.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster,
Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff,
Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika,
Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. A framework for few-shot
[language model evaluation, 12 2023b. URL https://zenodo.org/records/10256836.](https://zenodo.org/records/10256836)
Shangmin Guo, Biao Zhang, Tianlin Liu, Tianqi Liu, Misha Khalman, Felipe Llinares, Alexandre
Rame, Thomas Mesnard, Yao Zhao, Bilal Piot, et al. Direct language model alignment from online
ai feedback. arXiv preprint
Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan,
John Quan, Andrew Sendonaris, Ian Osband, et al. Deep q-learning from demonstrations. In
Proceedings of the AAAI conference on artifcial intelligence, volume 32, 2018.
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning
and compositional question answering. Conference on Computer Vision and Pattern Recognition
(CVPR), 2019.
Bo Li, Peiyuan Zhang, Kaichen Zhang, Fanyi Pu, Xinrun Du, Yuhao Dong, Haotian Liu, Yuanhan
Zhang, Ge Zhang, Chunyuan Li, and Ziwei Liu. Lmms-eval: Accelerating the development of large
[multimodal models, March 2024. URL https://github.com/EvolvingLMMs-Lab/](https://github.com/EvolvingLMMs-Lab/lmms-eval)
[lmms-eval.](https://github.com/EvolvingLMMs-Lab/lmms-eval)
Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang,
and Lingpeng Kong. Silkie: Preference distillation for large visual language models, 2023a.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Pope: Evaluating
object hallucination in large vision-language models, 2023b.
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr
Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European
conference on computer vision, pp. 740–755. Springer, 2014.
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. In The Twelfth International
Conference on Learning Representations, 2023a.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Llava-1.5: Improved baselines with visual
instruction tuning. arXiv preprint
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Llava: Visual instruction tuning, 2023c.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee.
[Llava-next: Improved reasoning, ocr, and world knowledge, January 2024. URL https://](https://llava-vl.github.io/blog/2024-01-30-llava-next/)
[llava-vl.github.io/blog/2024-01-30-llava-next/.](https://llava-vl.github.io/blog/2024-01-30-llava-next/)
Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, Bowen Zhang, Philipp Dufter,
Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, et al. Mm1: Methods, analysis & insights
from multimodal llm pre-training. arXiv preprint
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong
Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow
instructions with human feedback. Advances in neural information processing systems, 35:27730–
27744, 2022.
Yusu Qian, Haotian Zhang, Yinfei Yang, and Zhe Gan. How easy is it to fool your multimodal llms?
an empirical analysis on deceptive prompts. arXiv preprint
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea
Finn. Direct preference optimization: Your language model is secretly a reward model, 2023.
Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks.
In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing.
[Association for Computational Linguistics, 11 2019. URL https://arxiv.org/abs/1908.](https://arxiv.org/abs/1908.10084)
[10084.](https://arxiv.org/abs/1908.10084)
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object
hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods
in Natural Language Processing, pp. 4035–4045, 2018.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy
optimization algorithms. arXiv preprint
Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh,
and Marcus Rohrbach. Towards vqa models that can read. In Proceedings of the IEEE/CVF
conference on computer vision and pattern recognition, pp. 8317–8326, 2019.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford,
Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in
Neural Information Processing Systems, 33:3008–3021, 2020.
Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan,
Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Llava rhlf: Aligning large multimodal models
with factually augmented rlhf. arXiv preprint
Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov,
Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, and Will Dabney. Understanding
the performance gap between online and offline alignment algorithms, 2024a.
Yunhao Tang, Zhaohan Daniel Guo, Zeyu Zheng, Daniele Calandriello, Rémi Munos, Mark Rowland,
Pierre Harvey Richemond, Michal Valko, Bernardo Ávila Pires, and Bilal Piot. Generalized
preference optimization: A unified approach to offline alignment. arXiv preprint
2024b.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay
Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation
and fine-tuned chat models. arXiv preprint
Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement
learning. Machine learning, 8:229–256, 1992.
Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu,
and Yi Wu. Is dpo superior to ppo for llm alignment? a comprehensive study. arXiv preprint
Tianyu Yu, Jinyi Hu, Yuan Yao, Haoye Zhang, Yue Zhao, Chongyi Wang, Shan Wang, Yinxv Pan, Jiao
Xue, Dahai Li, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. Reformulating vision-language
foundation models and datasets towards universal multimodal assistants, 2023a.
Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu,
Hai-Tao Zheng, Maosong Sun, and Tat-Seng Chua. Rlhf-v: Towards trustworthy mllms via
behavior alignment from fine-grained correctional human feedback, 2023b.
Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He,
Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for
super gpt-4v trustworthiness. arXiv preprint
Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang,
and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv
preprint
Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason
Weston. Self-rewarding language models. arXiv preprint
Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, and Tong Sun.
Llavar: Enhanced visual instruction tuning for text-rich image understanding. arXiv preprint
Bo Zhao, Boya Wu, and Tiejun Huang. Svit: Scaling up visual instruction tuning, 2023a.
Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. Slic-hf:
Sequence likelihood calibration with human feedback. arXiv preprint
Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modalities in
vision large language models via preference fine-tuning, 2024.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint
A IMPLEMENTATION DETAIL
For all offline experiments, as well as for Online-DPO and Mixed-DPO, we conducted a hyperparameter search. The parameters included learning rates of 10 _[−]_ [7], 5 _·_ 10 _[−]_ [7], 10 _[−]_ [6], 5 _·_ 10 _[−]_ [6] ; projection
layer learning rates of 2 _·_ 10 _[−]_ [5], 2 _·_ 10 _[−]_ [6], 2 _·_ 10 _[−]_ [7] ; epochs of 3, 5, and 7; and batch sizes of 16, 32,
and 48. We reported the best results for each method. Additionally, we set the LoRA rank and scaling
factor to 128 and 256, respectively. The _β_ values for DPO, IPO, and SLiC were explored at 0 _._ 05, 0 _._ 1,
0 _._ 2, 0 _._ 5 for DPO; 0 _._ 8, 0 _._ 9, 1 _._ 0 for IPO; and 0 _._ 02, 0 _._ 1, 0 _._ 2 for SLiC.
For RL methods (PPO and RLOO), we maintained constant base model parameters while training
LoRA adapters for alignment. Specifically, for RLOO, we utilized _k_ = 4, generating four distinct
responses for each prompt at a temperature of 1.0. Training was conducted over two epochs with a
batch size of 256 and a learning rate of 3 _·_ 10 _[−]_ [4] . Prior to RLOO training, we calculated the mean
and standard deviation of rewards using the alignment dataset and normalized the rewards during
training to achieve zero mean and unit variance. We determined that a _β_ value of 0 _._ 4 provided the
best balance between rewards and the KL penalty for RLOO. Gradient clipping was also implemented
to cap the maximum gradient norm at 1.0.
For PPO specifically, we trained for 3 epochs with a learning rate of 1.41e-5 using a constant learning
rate schedule. We used 1 GPU with a batch of 32. For the reward model, we used a a learning rate of
2e-5 and trained for 4000 steps. The learning rate schedule was also adjusted to be constant but with
a warmup phase. The fraction value for the warmup phase is set at 0.03. Training was conducted on
8 GPUs with a batch size of 32.
B EVALUATION
B.1 POPE
We noticed the existence of an upper bound on the POPE benchmark, as most of our experiments
would reach a plateau between 86% and 88% despite improvements on other benchmarks. We
manually looked at the losses among 100 responses and present the results in this section.
In 20% cases, we observed that the ground truth was either incorrect or disputable. In some of
those cases, it appeared that the ontology used to build POPE could potentially result in differing
interpretations. For example, in certain countries, a clear distinction exists between a car and a truck,
although this distinction is not as pronounced in other regions of the world [8] . We provided an example
along the response of our aligned model [9] in Figure 4.
**Prompt:** Is there a tv in the image?
**POPE Ground Truth:** yes
**Aligned LLaVA 1.6-7B response:** no
Figure 4: Upon analysis of the losses on POPE, we noticed close to 20% of cases where the ground
truth was either incorrect or disputable. This example is from POPE, which sources images from
COCO (Lin et al., 2014).
[8An example of such distinction between car/truck can be seen on COCO_val2014_000000210789.jpg where](http://images.cocodataset.org/val2014/COCO_val2014_000000210789.jpg)
the POPE ground truth expects "no" to the prompt "Is there a car in the image?".
9We used a LLaVA 1.6-7B DPO-aligned on LLaVA prompts and responses sampled from VLFeedback. See
Section 5.3.
Provided these examples are eliminated, we think it is plausible that performant models could
potentially exceed a 90% accuracy rate on the POPE benchmark. Re-annotating those examples is
beyond the scope of this work, however we would like to invite the community to consider it as
many recent SOTA models exhibit such plateau. See Table 4 in (McKinzie et al., 2024) where all the
models reported are demonstrating such plateau on POPE.
B.2 CHAIR AND OBJECT HALBENCH
We evaluated two widely used benchmarks in the community for measuring hallucination, focusing
on the computation of CHAIR metrics. We investigated approaches described by Rohrbach et al.
(2018), which uses COCO annotations to compute CHAIR scores, and the more recent method by Yu
et al. (2023b), named Object HalBench, which combines COCO annotations with a GPT model
to enhance the detection of hallucinated objects.
Our analysis reveals that both benchmarks are significantly noisy (Figure 5). We also found that any
improvements in CHAIR scores strongly depend on the ability of these benchmarks to detect specific
types of hallucinations and cannot be attributed solely to the improvement of the model.
Furthermore, it is common to report CHAIR metrics without including recall metrics. Considering
the trade-off between CHAIR and recall, omitting recall does not provide a full picture of how much
a model has improved in reducing hallucinations. For instance, a model that generates short and
conscise responses might not produce many hallucinations, but this may be at the cost of potentially
providing an unhelpful answer.
Hence, the recall metric from Rohrbach et al. (2018) proves particularly informative for comparing
different models and helping with our understanding of other benchmarks. We report this metric in
our evaluations, styled Recall [coco] in our tables.
B.3 MMHALBENCH-VISION
The original MMHALBench benchmark (Sun et al., 2023) uses GPT-4 to judge whether model
responses introduce hallucinations. In that text-only regime, MMHALBench relies on ground truth
information about the pictures, such as the categories of the objects present or a human reference
response to the prompt.
We evaluated manually the common wins and losses obtained on MMHALBench during our experiments and noticed that in _∼_ 20% cases we disagree with the resulting MMHALBench score [10] . We
found cases where responses with hallucinations were considered as correct. Oppositely, we found
cases where valid answers were wrongly tagged as containing hallucinations. In many cases, we saw
the helpfulness to be under-estimated. See Figure 6.
This can be explained due the ground truth information being only expressed through text causing the
judge model, GPT-4, to wrongly tag or miss hallucinations. To mitigate such cases, we introduced
MMHALBench-Vision: we rely on the recently introduced GPT4-o to consume the image along
the text ground truth information. We kept the evaluation prompt and scoring identical.
C PREFERENCE DATA
C.1 DATASET PROMPT CORRUPTION
We leverage GPT-4 to corrupt chosen responses with realistic and plausible hallucinations. We call
_realistic hallucinations_ those instances where a human, just by looking at the corrupted response,
is unable to recognize it without having to refer back to the image. We have remarked this was
an important distinction: the more obvious the corruptions are, the poorer the performance of
the resulting policy is. We launched side experiments where we employed a less skilled LLM
corrupter and incorporated obvious tags [11] into the responses. In both scenarios, we noticed a drop in
1021 cases out of 96 while comparing wins and losses of two models.
11An example of obvious rejected response marked with a tag: _There are only two examples of plates that are_
_visible in the image._ _**This response is factually incorrect.**_
**Model Response:** The image features a brown and white cat lying
on its side in an orange suitcase. The cat appears to be relaxed and
comfortable in the suitcase. The suitcase is placed on a bed. The
suitcase is open, revealing its contents, which include a book and
a handbag. The cat is the main focus of the image, occupying a
significant portion of the frame. The orange suitcase and the handbag
are the other main objects in the scene, with the handbag being located
on the left side of the image and the suitcase occupying the right side.
The overall mood of the image is calm and cozy.
**Model Response:** The image features a large, colorful truck parked
on the side of a street. The truck is parked next to a brick building,
and there is a car parked behind it.
There are several people in the scene, with one person standing near
the truck and two others further away. One of the individuals is carrying a handbag, which is visible in the scene. The overall atmosphere
of the image is lively and engaging, with the eye-catching truck.
Figure 5: Examples illustrating instances where the CHAIR and Objet HalBench benchmarks
fail to detect hallucinations. Text highlighted in green identifies hallucinations successfully detected
by the benchmarks. In contrast, text highlighted in red indicates examples where the benchmark
failed to identify hallucinations. Orange indicates hallucinations that, though not targeted by these
benchmarks, degrade response quality. The top example shows the benchmark proposed by Yu et al.
(2023b) while the bottom example follows from (Rohrbach et al., 2018). Images are from COCO (Lin
et al., 2014).
performance as the corruption gets less realistic and readily ‘hackable’ by the policy under alignment.
The prompt used to corrupt the chosen responses is reproduced in Table 13.
C.2 DATASET SIZE ABLATION WITH THE CORRUPTING STRATEGY
We conducted a dataset size ablation on the application of our corrupting strategy on VLFeedback
(Figure 7) . We evaluated 7 checkpoints between 100 and 5,000 training samples, our maximum in
this data regime (Section 5.3). We provide the baseline results with a dashed line. While POVID
shows the best result on Recall [coco], our simple corruption strategy applied outperforms other datasets
on both LLavaBench-in-the-Wild and MMHALBench hallucination rate, while being on par
on the MMHALBench helpfulness rate with VLFeedback vanilla.
D ANNOTATOR IN ONLINE-DPO
Table 14 shows the prompt we used to obtain online feedback from the annotator. We conducted
multiple experiments with different prompts. In one setup, similar to the approach taken by
Guo et al. (2024) with the rewards model, we included the ground truth response as an additional
signal for the annotator to evaluate both responses. We did not observe any significant change in
either the evaluation metrics or the final performance of the aligned model. This may be due to
the fact that most of the open-source MLLMs we used in this study still lack the ability to follow
**Prompt:** How many people are there in the image?
[image ID: 16189396430_4dce91a9d7_o](https://huggingface.co/datasets/Shengcao1006/MMHal-Bench/blob/main/test_data.zip)
**MMHAL reference response:** There are four people in the image.
**MMHAL categories** : Girl, Human head, Human hair, Mammal,
Dog, Person, Human face
**LLaVA 1.6-7B aligned:** There are four people in the image: a man,
a woman, and two children.
**GPT-4 response:** total score=1, hallucination=1, helpfulness=1
**Prompt :** Compare the two different views of the backpack.
[image ID: 12392564684_ffd19b5b55_o](https://huggingface.co/datasets/Shengcao1006/MMHal-Bench/blob/main/test_data.zip)
**MMHAL reference response:** The image shows two different views
of the same black backpack. On the left side, it shows the inside of
the bag, containing a camera, two lenses an other accessories. On the
right side, it shows the outside of the bag, and there is a "Lowepro"
logo on it.
**MMHAL categories** : Hiking equipment, Clothing, Backpack
**Aligned LLaVA 1.6-7B response:** The image shows a backpack
with two different views: the top view and the side view. In the top
view, we can see (...)
**GPT-4 response:** total score=4, hallucination=0, helpfulness=1
Figure 6: Example of incorrect assessments with MMHALBench. The top example shows a response
incorrectly assessed as having a hallucination. The second presents a response where the model
hallucinates a side view of the backpack, but that is missed by MMHALBench. In both cases, the text
ground truth does not give enough information to the text-only evaluator to evaluate the response.
Reproduction of MMHALBench images cannot take place here due to usage restrictions. Please refer
[to https://huggingface.co/datasets/Shengcao1006/MMHal-Bench.](https://huggingface.co/datasets/Shengcao1006/MMHal-Bench)
instructions effectively, especially when the instructions contain multiple components or detailed
steps.
We also examined the potential bias of the annotator model in choosing "Response 1" or "Response
2" and found no noticeable bias.
Figure 8 shows an example of an annotation made by LLaVA 1.6-34B model.
E BIAS-DRIVEN HALLUCINATION SAMPLING
E.1 ADDING NOISE TO IMAGES
This section describes how to gradually add noise to images through a diffusion process. The
derivation follows the public implementation of POVID-style image distortion (Zhou et al., 2024) to
enable the proper reproduction of their results.
Let _x_ img( _k_ ) denote the image after applying noise _k_ -times with _x_ img(0) referring to the original image
and _N_ (0 _,_ 1) represent the normal distribution. Then the forward noise process is defined as:
_x_ img( _k_ ) = ~~�~~
1 _−_ _βkx_ img( _k −_ 1) + ~~�~~
_βkϵ_ with _ϵ ∼N_ (0 _,_ 1) _._ (8)
Hereby, _βk_ denotes a time-variant parameter which is set to _βk_ = _σ_ ( _−_ 6 + 1000 [12] _[k]_ [)] _[ ·]_ [ (0] _[.]_ [5] _[ ·]_ [ 10] _[−]_ [2] _[ −]_
10 _[−]_ [5] ) + 10 _[−]_ [5] to gradually increase noise between _k_ = 0 and _k_ = 1000 (refer to Figure 9).
The recursive equation (8) can be reformulated to apply _k_ steps of noise at once. Setting _αk_ = 1 _−_ _βk_
and ¯ _αN_ = [�] _k_ _[N]_ =1 _[α][k]_ [, the following equation applies] _[ N]_ [ steps of noise to image] _[ x]_ [img][(0)][:]
_√_
_x_ ˜img( _N_ ) = _[√]_ _α_ ¯ _N_ _x_ img(0) +
1 _−_ _α_ ¯ _N_ _ϵ_ with _ϵ ∼N_ (0 _,_ 1) _._ (9)
The default for _N_ in Zhou et al. (2024) is _N_ = 500.
Table 13: Prompt used to corrupt datasets with GPT-4.
E.2 ENSURING SEMANTICALLY MEANINGFUL DIFFERENCES
Section 3.4.3 describes an iterative technique for BDHS that evaluates similarity scores between the
generated response ˜ _y_ _[−]_ and the ground truth _y_ [+] . If both responses are identified as similar according
to the sentence embeddings model, a new BDHS response is sampled until a maximum number of
iterations _N_ BDHS is reached. The last iteration waives the ground truth reference and generates a full
response which is then taken as ˜ _y_ _[−]_ regardless of the similarity score. Figure 10 shows the number
of non-similar responses, i.e. ensuring _ϵ_ s _<_ 0 _._ 97, over the number of BDHS iterations for the full
POVID (5k) dataset. As expected all BDHS variants result in a larger number of non-similar responses
compared to the model response without image attention blocking or noisy images. Running BDHS
with a single iteration already results in more than 74 % semantically different responses. After four
|Col1|Col2|Col3|Col4|Col5|Col6|Col7|
|---|---|---|---|---|---|---|
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
|POVID|||||||
|VLFeedback|||||||
|VLFeedback
|Corrupted
|(Ours)|||||
|LLaVA 1.6-|B||||||
Preference training set size
|Col1|Col2|Col3|Col4|Col5|Col6|Col7|Col8|
|---|---|---|---|---|---|---|---|
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|||||||||
|P
|VID
|||||||
||LFeedbac
LFeedbac|Corrupted|(Ours)|||||
|||||||||
|L|LaVA 1.6-|B||||||
|||||||||
Preference training set size
1 _._ 4
1 _._ 3
1 _._ 2
1 _._ 1
|Col1|Col2|Col3|Col4|Col5|Col6|Col7|
|---|---|---|---|---|---|---|
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
|P|VID||||||
|V|LFeedback||||||
|V
|LFeedback
|Corrupted (
|Ours)||||
|L|LaVA 1.6-7|B|||||
Preference training set size
|Col1|Col2|Col3|Col4|Col5|Col6|Col7|
|---|---|---|---|---|---|---|
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
||||||||
|P
|OVID
||||||
||LFeedbac
LFeedbac|Corrupted|(Ours)||||
|
|
LaVA 1.6-|
 7B|||||
||||||||
Preference training set size
0 _._ 5
0 _._ 45
0 _._ 4
0 _._ 35
0 _._ 3
Figure 7: Impact of the preference dataset size. Our corrupting strategy outperforms other datasets on
LLaVA-in-the-Wild and MMHALBench hallucination rate. It is on par on the MMHALBench
helpfulness rate against vanilla VLFeedback. Finally, POVID reports the highest Recall [coco] . The
dashed lines are the scores for the LLaVA 1.6-7B baseline.
iterations, BDHS variants with restricted image access differ in over 90 % while the last iteration is
guidance free and only depends on the sampled attention mask resp. noise. Interestingly, in iteration 5,
BDHSattn _, ρ_ th=1 corresponds to guidance-free response generation with fully blocked image tokens
which still results in 7 % similar responses w.r.t. the SFT ground truth. Probable reasons for this
saturation are either that the correct answer is easy to guess even without access to the image, or that
the answer is memorized from the training data. Note that prompts and images in POVID (5k) are
extracted from LLaVA Instruct which served as training data for fine-tuning LLaVA-1.6.
BDHS with noisy images in the input and _N_ = 500 diffusion steps results in more than 99 %
semantically different responses after five iterations, surpassing the score for the fully blocked
response. This is misleading, as although the responses are indeed semantically different, they mostly
mention that the prompt cannot be evaluated due to blurry and noisy images. Essentially, the noise
adds an additional bias towards noise/pixel-referring responses instead of inducing only the desired
inherent bias which would saturate at approx. 93 % (response with fully masked image tokens).
Table 14: The Prompt Used for the Annotator.
**Prompt** How many people are in the pool?
**Response 1:** There is a person in the pool, a young man
enjoying his time surfing the small waves.
**Response 2:** This is not a swimming pool but the ocean, and
on the ocean, there is a person. A young man is enjoying his
time surfing.
**Annotator:** Based on the image content, Response 2 is better
and more helpful. The image shows a person surfing in the
ocean, not a swimming pool. The surfer is riding a wave, and
there are no other people visible in the image. The description
accurately reflects the content of the image and does not include any hallucinations or irrelevant details.
Figure 8: Example of Annotation. Image is from COCO (Lin et al., 2014).
0 _._ 006
0 _._ 004
0 _._ 002
0 100 200 300 400 500 600 700 800 900 1 _,_ 000
Step _k_
Figure 9: Schedule of diffusion parameter _βk_ .
After three iterations, the score of BDHSattn _, ρ_ th=0 _._ 99 reaches the one from the fully blocked response
which is hypothetically implied due to increased diversity by subsampling a distinct attention mask.
Section E.5 presents several examples with actual responses.
E.3 BDHS ALGORITHM
The general overview of BDHS is provided in Figure 2. This section introduces the corresponding
algorithm which is listed in Algorithm 1. This version includes both, noisy images for BDHSnoise and
attention masking for BDHSattn (refer to the comments in Algorithm 1). We add a straightforward
heuristic to swap _yes_ and _no_ words whenever they occur in the beginning of a sentence. For this
purpose line 12 introduce a regular expression which matches any _yes_ or _no_ at the beginning of each
sentence and optionally skips any preceding newline or whitespace characters. This expression can
be extended to further use-cases if desired. We choose to generate the full response without any SFT
BDHSnoise _, N_ =300 BDHSnoise _, N_ =500 BDHSattn _, ρ_ th=0 _._ 98
BDHSattn _, ρ_ th=0 BDHSattn _, ρ_ th=0 _._ 99 BDHSattn _, ρ_ th=1
1 2 3 4 5
BDHS Iteration
Figure 10: Number of resolved similar responses for BDHS generation based on POVID (5k).
Parameters are _ϵ_ s = 0 _._ 97 and _N_ BDHS = 5.
ground truth guidance in the very last iteration whenever _N_ BDHS _>_ 1 to minimize similarity (refer to
line 5).
**Algorithm 1** BDHS
**Require:** Prompt _x_ text, image _x_ img, SFT ground truth _y_ [+], attention masking parameter _ρ_ th, image noise level _N_,
BDHS iterations _N_ BDHS, similarity threshold _ϵ_ s
1: **for** _i_ = 1 _,_ 2 _, . . ., N_ BDHS **do**
2: _m ←_ Sample image attention mask with _ρ_ th according to (6) _▷ρ_ th _>_ 0 only for BDHSattn
3: _x_ ˜img _←_ AddNoise( _N, x_ img) via (9) _▷N >_ 0 only for BDHSnoise
4: _x_ ˜ _←_ ( _x_ text _,_ ˜ _x_ img _, m_ )
5: **if** _N_ BDHS _>_ 1 and _i_ = _N_ BDHS **then**
6: _y_ _[−]_ _←_ Generate full model response via _πθ_ ( _·|x_ ˜)
7: **return** _y_ _[−]_
8: _S ←_ Split _y_ [+] into S sentences
9: _yk_ _[−]_ _[←∅]_ _▷_ Initialize empty string
10: **for each** _yk_ [+] [in] _[ S]_ **[ do]** _▷_ Parallelizable
11: _ξ ←_ _ξ ∼U_ (0 _,_ 1) _▷_ _U_ (0 _,_ 1) denotes the uniform distribution in [0 _,_ 1]
12: **if** _yk_ [+] [matches] _[ r"ˆ[\s]]_ _*_ _[(Yes|yes|No|no)"]_ [ and] _[ ξ][ ≥]_ [0] _[.]_ [5] **[ then]** _▷_ _r"·"_ denotes a regular expression
13: _yk_ [+] _[←]_ [Swap corresponding] _[ Yes]_ [/] _[yes]_ [ by] _[ No]_ [/] _[no]_ [ and vice versa]
14: _yk,_ [+] 1 _[←]_ [Sample random position in] _[ y]_ _k_ [+] [and return first substring]
15: _yk,_ _[−]_ 2 _[←]_ [Complete sentence via (7) until full stop or] _[ <eos>]_
16: _yk_ _[−]_ _[←]_ [(] _[y]_ _k,_ [+] 1 _[, y]_ _k,_ _[−]_ 2 [)] _▷_ Concatenate strings to full sentence
17: _y_ _[−]_ _←_ ( _y_ _[−]_ _, yk_ _[−]_ [)] _▷_ Append to overall response
18: _ϕ ←_ Compute similarity score between _y_ _[−]_ and _y_ [+] in [0 _,_ 1] _▷_ Use sentence embeddings
19: **if** _ϕ < ϵ_ s **then**
20: **break** _▷_ Semantically different according to threshold
21: **return** _y_ _[−]_
E.4 ADDITIONAL ABLATIONS
Additional BDHS ablations, especially regarding different hyperparameter choices are shown in
Table 15. We also evaluate SFT guidance-free generation only with attention masking active. The
corresponding benchmark results are listed in the first two rows. All subsequent rows evaluate the
full BDHS approach including SFT guidance. We include ablations that rely on noisy images rather
than attention masking, following the diffusion process described in E.1.
E.5 ADDITIONAL EXAMPLES
This section presents further examples of responses generated from LLaVA instruct prompts and
images. The different variants of BDHS are introduced in Section 3.4. Refer to Figure 11 for the
|y˜−derived from policy|POPE↑ MMHAL↑ MMHALV ↑ LLaVAW↑ VQAT ↑ GQA↑ MMVet↑ Recallcoco ↑|
|---|---|
|– (Baseline)
– (Plain DPO)|86.40
**2.95**
2.75
80.85
64.85
64.23
**43.94**
68.13
88.18
2.93
**2.93**
81.89
64.90
64.34
43.39
71.80|
|Attention Masking,_ ρ_th = 0_._98
88.61
2.25
2.25
82.25
64.92
64.04
42.75
**77.46**
Attention Masking,_ ρ_th = 0_._99
88.70
2.52
2.51
86.08
65.07
64.06
42.02
77.04|Attention Masking,_ ρ_th = 0_._98
88.61
2.25
2.25
82.25
64.92
64.04
42.75
**77.46**
Attention Masking,_ ρ_th = 0_._99
88.70
2.52
2.51
86.08
65.07
64.06
42.02
77.04|
|BDHSattn,_ ρ_th = 0_._98
**88.80**
2.56
2.68
**86.54**
65.02
64.03
43.03
76.10
BDHSattn,_ ρ_th = 0_._99
88.75
2.61
2.71
86.33
65.07
63.97
43.39
75.58
BDHSattn,_ ρ_th = 1_._00
88.70
2.63
2.80
84.15
**65.18**
63.93
43.12
75.37|BDHSattn,_ ρ_th = 0_._98
**88.80**
2.56
2.68
**86.54**
65.02
64.03
43.03
76.10
BDHSattn,_ ρ_th = 0_._99
88.75
2.61
2.71
86.33
65.07
63.97
43.39
75.58
BDHSattn,_ ρ_th = 1_._00
88.70
2.63
2.80
84.15
**65.18**
63.93
43.12
75.37|
|BDHSnoise,_ N_ = 100
88.50
2.58
2.48
82.46
64.96
**64.34**
40.14
75.47
BDHSnoise,_ N_ = 200
88.55
2.49
2.38
83.43
65.10
64.24
38.76
74.53
BDHSnoise,_ N_ = 300
88.59
2.43
2.45
85.16
65.11
64.18
40.69
76.10
BDHSnoise,_ N_ = 400
88.66
2.39
2.42
83.72
65.09
64.29
40.41
75.16
BDHSnoise,_ N_ = 500
88.59
2.36
2.49
84.53
65.05
64.14
41.38
75.16|BDHSnoise,_ N_ = 100
88.50
2.58
2.48
82.46
64.96
**64.34**
40.14
75.47
BDHSnoise,_ N_ = 200
88.55
2.49
2.38
83.43
65.10
64.24
38.76
74.53
BDHSnoise,_ N_ = 300
88.59
2.43
2.45
85.16
65.11
64.18
40.69
76.10
BDHSnoise,_ N_ = 400
88.66
2.39
2.42
83.72
65.09
64.29
40.41
75.16
BDHSnoise,_ N_ = 500
88.59
2.36
2.49
84.53
65.05
64.14
41.38
75.16|
Table 15: Additional ablation results for Offline-BDHS. All results are based on LLaVA 1.6-7B,
using DPO and the POVID (5k) sample for the source of images and prompts. and prompt.
examples. For guided generation, the colored text is generated purely from the model while the
standard text is taken from the SFT ground truth.
Figure 11: Examples of generated responses from BHDS ablations and POVID-style image distortion.
The image, prompt and SFT ground truth are taken from LLaVA-Instruct-150k, which sources them
from CoCo (Lin et al., 2014).


Mohammad Saeed Ebrahimi Saadabadi _[∗]_, Sahar Rahimi Malakshan _[∗]_,
Ali Dabouei, and Nasser M. Nasrabadi
West Virginia University
_∗_ Equal Contribution
{me00018, sr00033,Ad0046}@mix.wvu.edu, nasser.nasrabadi@mail.wvu.edu
**Abstract.** Aiming to enhance Face Recognition (FR) on Low-Quality
(LQ) inputs, recent studies suggest incorporating synthetic LQ samples
into training. Although promising, the quality factors that are considered
in these works are general rather than FR-specific, _e.g.,_, atmospheric
turbulence, resolution, _etc_ . Motivated by the observation of the vulnerability of current FR models to even small Face Alignment Errors (FAE)
in LQ images, we present a simple yet effective method that considers FAE as another quality factor that is tailored to FR. We seek to
improve LQ FR by enhancing FR models’ robustness to FAE. To this
aim, we formalize the problem as a combination of differentiable spatial transformations and adversarial data augmentation in FR. We perturb the alignment of the training samples using a controllable spatial
transformation and enrich the training with samples expressing FAE. We
demonstrate the benefits of the proposed method by conducting evaluations on IJB-B, IJB-C, IJB-S (+4.3% Rank1), and TinyFace (+2.63%).
[https://github.com/msed-Ebrahimi/ARoFace](https://github.com/msed-Ebrahimi/ARoFace)
**Keywords:** Unconstrained Face Recognition · Face Alignment · LowQuality Input · Adversarial Data Augmentation

ARoFace: Alignment Robustness to Improve Low-Quality Face Recognition

Mohammad Saeed Ebrahimi Saadabadi
Sahar Rahimi Malakshan
Ali Dabouei
Nasser M. Nasrabadi


Excellence in Face Recognition (FR) is attributed to large-scale training datasets
[21,92], advanced deep networks [22,24,57], and angular criteria [11,36,42,62].
However, current FR models experience performance failure in practical scenarios where images are acquired from long-range distances, _I.e.,_ Low-Quality
(LQ) or unconstrained images [2, 3, 5, 31, 54]. Specifically, the reported performance [41, 55] of the state-of-the-art (SOTA) models on IJB-S [31] and TinyFace [5], _I.e.,_ LQ benchmarks, are about 30% lower than LFW [25], _I.e.,_ HighQuality (HQ) benchmark. This deficiency stems from the distribution gap between the training and LQ testing data, _I.e.,_ lack of a sufficient number of LQ
instances during training [18,41,54,58,65,84]. An intuitive solution would be to
construct a large-scale dataset with an adequate amount of LQ samples which
is impractical due to data acquisition costs and privacy concerns [41,60].
2 Saadabadi et al.
**Fig. 1:** Visual comparison of aligned (a) and alignment-perturbed (b) samples from the
IJB-B dataset. (c, d, e) The performance difference between aligned inputs and those
with slight FAE. Models exhibit robustness to FAE in HQ samples but suffer significant
performance drops in LQ faces, with over 50% reduction in TAR@FAR = 1 _e−_ 5. Results
from two distinct ResNet-100 trained on MS1MV3 using ArcFace/AdaFace objective.
Considering the components of a practical FR system, _I.e.,_ face detection,
face alignment, and face recognition model [70, 86], available LQ FR methods
primarily have focused on how LQ inputs affect the recognition model [4, 18,
23, 40, 41, 56, 58, 65, 68, 78]. These studies can be categorized into Facial Image
Enhancement (FIE) [4, 20, 23, 40, 43, 68, 78, 83], and Common Space Mapping
(CSM) [18, 41, 56, 58, 65, 84]. FIE estimates HQ face from LQ counterpart and
then performs FR, yet faces the challenge of ill-posedness, _I.e.,_ multiple HQ
outcomes exist for a single LQ face [26,56,75,80]. CSM studies [18,56,58,65,84]
seek to map LQ and HQ instances to a common embedding by employing image
degradations as data augmentation during training. However, recent CSM works
lack practicality due to deficiency in expressing real-world LQ [3, 47, 47, 48],
entailing high computational resources [50], identity preservation challenge [1,
39,56,66,76], and constrained of _a priori_ defined target data [48,87].
We extend our focus beyond the face recognition model to explore the impact
of LQ inputs on face detection and alignment. It has been shown that face
detector accuracy diminishes with LQ inputs, leading to Face Alignment Error
(FAE) [10,35,59,70,85]. We investigate the impact of FAE on FR performance by
manually adding small random spatial perturbations including scaling, rotation,
and translation, to the original aligned evaluation faces, as shown in Figures 1a,
and b. The results in As depicted in Figures 1c, d, and e, the performance gap
of a given model between alignment-perturbed and aligned samples. Based on
these observations, we found that while SOTA FR networks are robust against
FAE in HQ faces, they are susceptible to FAE in LQ samples, _e.g.,_, more than
50% drop in True Acceptance Rate (TAR) at a False Acceptance Rate (FAR) of
1 _e −_ 5 (TAR@FAR=1e-5) on IJB-B and IJB-C.
Therefore, we contend that FAE plays a crucial role in the failure of FR
models on LQ faces. Current CSM methods [18, 41,56,84] enrich training with
synthetic LQ faces yet ignore the FAE as another degradation component. Consequently, the current FR models are extremely susceptible to the intersection
of FAE and LQ input. A potential remedy is end-to-end training of both recognition and detection networks. However, this is limited by high computational
demands and the lack of large datasets with both types of labels. A few studies
tried to establish alignment-free FR by incorporating an alignment module into
the recognition model [28,70,88,89]. However, the estimated spatial transforma
ARoFace for Low-Quality Face Recognition 3
tion expresses coarse geometric information as a holistic parametric model and
falls short of improving practical LQ FR [35,41,54].
In an orthogonal direction, we consider FAE as another image degradation
factor and aim to develop an FR model robust against FAE, dubbed Alignment
Robust Face (ARoFace) recognition. Our proposal is inspired by the adversarial
data augmentation that enhances model robustness through integrating adversarial components into the training process [19,72]. Concretely, ARoFace employs
a differentiable spatial transformation [28] to adversarially perturb the alignment
of training samples and enrich the training with samples expressing FAE.
We craft samples expressing FAE by applying spatial transformations including scaling, rotation, and translation, to the benign (aligned) sample. We employ
a global transformation grid [28] that avoids face deformation to compute faithful
and beneficial instances with meaningful spatial variations. Moreover, a randomized adversarial budget is used to improve the diversity of the crafted samples,
helping to attain a variety of spatial transformations for a single input.
Compared to recent face degradation methods [41,54] our approach imposes
negligible trainable parameters to the training pipeline, _I.e.,_ a maximum of
nine additional parameters. Moreover, the proposed method avoids the class
distortion problem, _I.e.,_ changing the identity in the output face, of GANbased face degradation by not altering the intensity of the faces and merely
modifying the position of pixels [1,39,56,66,76]. Thus, the resulting misaligned
face completely lies in the manifold of faithful faces for the FR training [19,48,91].
Furthermore, ARoFace circumvents the two-stage optimization and the necessity
of accessing target data in previous GAN-based studies [41,54]. It is worth noting
that our method does not require any face or landmark detection module, and
can be readily integrated into the training pipeline of an arbitrary FR model to
improve the generalization across LQ evaluations. In summary, the contributions
of the paper are as follows:
**–** We introduce FAE as an image degradation factor tailored for FR which has
previously been ignored in LQ FR studies.
**–** We propose an optimization method that is specifically tailored to increase
the FR model robustness against FAE.
**–** We show that the proposed optimization can greatly increase the FR performance in real-world LQ evaluations such as IJB-S and TinyFace. Moreover,
our framework achieves these improvements without sacrificing the performance on datasets with both HQ and LQ samples such as IJB-B and IJB-C.
**–** We empirically show that the proposed method is a plug-and-play module,
providing an orthogonal improvement to SOTA FR methods.


Approaches to tackle LQ FR can be categorized into: 1) Facial Image Enhancement (FIE) [4,30,43,83], and 2) Common Space Mapping (CSM) [18,58,65,84].
4 Saadabadi et al.
FIE methods try to solve the inverse problem of retrieving HQ samples from
their LQ counterparts and then perform FR [26, 75, 80]. Despite the success in
enhancing visual quality metrics such as PSNR [67], SSIM [67], and LPIPS [29],
they fail to increase the FR performance on real-world testing benchmarks since
estimating HQ face from an LQ counterpart is an ill-posed problem, _I.e.,_ multiple HQ faces exist for single LQ input [34,41,79]. Conventional CSM methods
try to find joint embedding for (HQ, LQ) pairs using sample-wise supervision.
However, these approaches suffer from convergence issues when integrated into
large-scale FR training framework [64].
Recently, Shi _et al_ . proposed to employ an unlabeled LQ face dataset to
establish a GAN as an image degradation module. In the follow-up work [41],
Liu _et al_ . incorporate feedback from the FR network to the degradation so
it can control the amount of image degradation to be applied to each samples.
However, the requirement of _a priory_ selected target dataset limits their practical
usability [48]. Moreover, GANs are cumbersome to deploy during large-scale
training and can introduce class distortion issues [56,66,76]. Also, developing a
proper generative model heavily relies on prior knowledge [1,39,65,76].


Employing adversarial images as additional training data has been extensively
surfed in various deep learning applications [19,38,44,72,73]. Broadly, adversarial data augmentation in FR intends to enhance the model’s resilience against
certain adversarial components in the inputs [7, 8, 37, 52, 82]. Sharif et al. [52]
devised real-world adversarial samples by adding printed glasses to the face images, while Komkov _et al_ . [37] explored adversarial hats. Deb _et al_ . [8] initially leveraged GANs for synthesizing adversarial samples. Subsequently, Yin
_et al_ . [82] introduced an adversarial makeup generation framework, while Dong
_et al_ . [15, 16] harnessed generative models for creating adversarial attributes.
Dabouei _et al_ . employed spatial transformation on facial landmarks to produce
geometrically-perturbed adversarial faces. Liu _et al_ . [41] employed the domaintranslation property of GAN combined with adversarial training to boost the
LQ FR. However, the necessity of data collection and pre-defined target data
make these methods extremely specific to a particular scenario and reduce the
practicality.

2.2 Adversarial Data Augmentation in Face Recognition


The pioneering work of Jaderberg _et al_ . [28] introduced a differentiable layer
that performs Spatial Transformation (ST) to an input image or feature map.
Owning to its intuitive transparency, the ST layer has inspired a multitude of
subsequent studies in applications including, but not limited to, classification,
dense correspondence matching, and FR [33,74,90].
Face alignment focuses on matching faces to a unified template, thereby
minimizing variations in geometry that are unrelated to identity. This typically
involves a 2D affine transformation to map facial landmarks to a specific 2D
ARoFace for Low-Quality Face Recognition 5
adversarial and original samples, _I.e.,_ minimization of _L_ .
template [12, 42, 61, 63]. Efforts to create an end-to-end FR system have led
to investigations into learning flexible, non-rigid transformations for establishing an alignment-free FR system, _I.e.,_ integrating alignment into recognition
module [70,89]. The approach by Wu _et al_ . [70] involved a recursive spatial transformer for complex transformation learning. Similarly, Zhou _et al_ . [89] employed
locally estimated homography transformations through a rectification network
for face correction. These techniques aim to achieve alignment independence by
concurrently learning alignment with the recognition process in a seamless, endto-end manner. Taking a different approach, Xu _et al_ . [74] explored optimizing
the target template for alignment, rather than adjusting alignment parameters.
However, their practical use is often hindered by computational demands and a
potential decrease in the discriminatory capacity of the recognition network [35].

2.3 Face Alignment and Learnable Spatial Transformation


In this paper, we use lowercase letters ( _e.g.,_, _x_ ) to denote scalars, lowercase
boldface ( _e.g.,_, **x** ) to denote vectors, uppercase letters ( _e.g.,_, _X_ ) to denote
functions, and uppercase calligraphic symbols ( _e.g.,_, _X_ ) to denote sets.
Let _D_ = _{_ ( **x** _n, yn_ ) _}_ _[n]_ _n_ [0] =1 [be the training dataset, consisting of] _[ n]_ [0][ aligned faces]
from _c_ classes. Furthermore, _Fψ_ = _C ◦_ _E_ denotes a deep neural network with
trainable parameter _ψ_ = [ _ψ_ 1 _, ψ_ 2] where _Eψ_ 1( _·_ ): R _[q]_ _→_ R _[d]_ is the deep feature
extractor that maps an input face **x** _∈_ R _[q]_, to a _d_ -dimensional representation
**z** _∈_ R _[d]_, and _Cψ_ 2 : R _[d]_ _→_ R _[c]_ is a parametric classifier, _I.e.,_ conventionally hyperspherical classifier in FR [11, 42, 62], with parameter _ψ_ 2 that maps **z** to a
probability distribution over _c_ classes. For brevity and convenience of presentation, all representations are _ℓ_ 2-normalized.


We propose to enhance LQ FR by increasing robustness to Face Alignment Error
(FAE) since FAE in LQ images is inevitable [10, 35, 59, 70, 85] and current FR
6 Saadabadi et al.
models are severely susceptible to FAE, as discussed in Section 1. To this end, the
proposed method considers FAE as another image quality factor, _I.e.,_ tailored
to FR, and leverages adversarial data augmentation combined with differentiable
spatial transformation to enrich the training with samples expressing FAE. In the
next two sections, we introduce adversarial data augmentation and differentiable
spatial transformation, respectively, as prerequisites to the proposed method.
Then, we provide a detailed description of our method.
**Adversarial Data Augmentation** aims to improve training without requiring
additional data [7,44,49]. Specifically, it seeks to craft hard training instances
(adversarial samples) from the original sample **x** (benign):
\ lab el _e_ q _:a_ d _at_             - **t** a _l }_ _\theta ^{*} = \argmax _{{\theta }}L(F_{\psi }; T_{\theta }(\bx ), y),_ (1)
_{_
where _Tθ_ ( _._ ) is an augmentation function with parameter _θ_, and _L_ is the learning
objective, _I.e.,_ empirical risk. This optimization can be solved through _k_ -steps
Projected Gradient Descent (PGD) [44]:
\ _l_ abe _l_ {or _i_ g _i_ n _a_ l p _g_ d} \ _be_ g _i_ n { _al_ i _gn_ e _d_ } **\** t _h e_ ta __{(k+1)} \gets \proj _{\mathcal {S}} \left ( \theta _{(k)} + \alpha \sgn \left (\nabla _{\theta _{(k)}} L(F_{\psi };T_{\theta _{(k)}}(\bx ), y) \right )\right ), \end {aligned}_ (2)
where sgn( _._ ) denotes the sign function, _α_ is the PGD step size, _S_ is set of allowed
perturbation and proj _S_ ( **s** ) projects **s** back into _S_, proj _S_ ( **s** ) = arg min **s** _[′]_ _∈S_ =
_||_ **s** _−_ **s** _′||_ 2. Typically, _S_ is defined as _ℓp_ -norm ball center at **x** with radius _ρ_ :
\ _la b el_ _{_ e **q** : _s_ **p** _ace }_ _\begin {aligned} \mathcal {S}=\{\theta \:| \: \:\:||T_{\theta }(\bx )-\bx ||_{p}\leq \rho \}. \end {aligned}_ (3)
**Spatial Transformation.** Here, we detail the spatial transformation applied
to every channel of input data, **x** _∈_ R _[q]_, where _q_ is 3 _× h × w_ for RGB input. Let
the row and column indices ( _i, j_ ) be as points ( _u, v_ ) _∈_ R [2], where the _u_ -axis and
the _v_ -axis are horizontal and vertical axis, respectively. Furthermore, _Pu_ ( _j_ ) =
_j −_ _[w][−]_ 2 [1] and _Pv_ ( _i_ ) = _[h][−]_ 2 [1] _−_ _i_ convert zero-indexed ( _i, j_ ) to _u, v_ coordinates. _Tθ_
is an invertible affine transformation with parameter _θ_ = ( _φ, ∆u, ∆v, λ_ ), where
_φ ∈_ [0 _,_ 2 _π_ ] denotes the rotation angle, _∆u ∈_ R, _∆v ∈_ R denote horizontal and
vertical shifts, respectively, and _λ ∈_ R and _λ > −_ 1 denotes the scaling factor [77].
For each location ( _Pu_ ( _j_ ) _, Pv_ ( _i_ )) the coordinate that maps to this location under
_Tθ_ can be obtained as:
_\_ _[l]_ _a b_ _[e]_ l _{o_ _[f]_ [l] w _fi_ e _l_ d _r eq_ u _i_ rm _ent} \small \begin {aligned} (\haxis ', \vaxis ') = T^{-1}_{\theta }(P_{\haxis }(j),P_{\vaxis }(i)), \end {aligned}_ (4)
where _Tθ_ _[−]_ [1] is the inverse transformation. ( _u_ _[′]_ _, v_ _[′]_ ) may not align with integervalued pixel indices in the input image, hence, we can utilize the bi-linear interpolation kernel of Jaderberg _et al_ . [28]:
_er_ p
- _ion}_ I_{\i _m_ g _}( \ ha_ x _i_ s, \vaxi _s_ _) =_ _\s_ u _m_ ___ { _i=0}^{h -1}\sum _{j=0}^{w -1} {\img }_{i,j}\cdot \max (0,1-|\vaxis -P_{\vaxis }(i)|)\cdot \max (0,1-|\haxis -P_{\haxis }(j)|),_ (5)
_l_ at
_\_ l _ab e_ l
_{e_ q
:
_i_ nt
ARoFace for Low-Quality Face Recognition 7
**Algorithm 1** ARoFace
**Input:** Dataset _D_, _f_ for conventional five landmarks in FR, number of PGD steps
_k_, distribution _Θ_ for _θ_, distribution _N_ for _α_, and the total training iteration _t_
**Output:** FR network _Fψ_
1: Initialize _Fψ_
2: **for** _i_ = 0 _. . . t −_ 1 **do** _\triangleright_ outer optimization
3: Sample _B_ from _D_
4: Sample _θ ∼_ _Θ_
5: Sample _α ∼_ _N_
6: **for** _j_ = 1 _. . . k_ **do** _\triangleright_ inner optimization for _k_ steps
7: Compute _l_ = _L_ ( _Fψ_ ; _Tθ_ ( _B_ **x** ) _, By_ )
8: Update _θj_ +1 _←_ _θj_ + _α_ sgn( _∇θj_ ( _l_ ))
9: _θ_ _[∗]_ _←_ proj _S_ ( _θj_ +1)
10: **end for**
11: Compute _l_ 1 = _L_ ( _Fψ_ ; _Tθ_ _[∗]_ ( _B_ **x** ) _, By_ ) _\triangleright_ FR objective alignment-perturbed faces
12: Compute _l_ 2 = _L_ ( _Fψ_ ; _B_ **x** _, By_ ) _\triangleright_ FR objective aligned faces
13: Update _ψ ←_ _ψ −_ _η∇ψ_ ( _l_ 1 + _l_ 2)
14: **end for**
where _xi,j_ denotes pixel value in the _i_ -th row and _j_ -th column of an arbitrary
_′_
channel in **x** . Thus the value of each pixel in the transformed image **x** is:
_[\]_ _lab_ [e] _[ l]_ { _eg_ _[q]_ [:] e _om_ e _t_ r _i c-_ p _e_ rtu _rbation} {\img }_{i,j}^\prime = I_{\img } \big (T^{-1}_{\theta } (P_{\haxis }(j),P_{\vaxis }(i))\big ),_ (6)
detailed derivation can be found in [28]. With a slight abuse of notation, we term
**x** _[′]_ = _Tθ_ ( **x** ) as the transformed version of **x** under the transformation _Tθ_ .


As observed in Figure 1, conventional FR networks are susceptible to even mild
Face Alignment Errors (FAE) in LQ faces. This vulnerability is a primary cause
for LQ FR failure since FAE is inevitable in LQ images [10,35,59,70,85]. Previous
LQ FR works [4,18,41,54,65,83,84] have ignored FAE and solely tried to make
the FR model robust against general image quality factors that are not tailored
to FR, _e.g.,_, resolution, atmospheric turbulence, blur, _etc_ . We consider FAE
as another degradation component and seek to improve LQ FR performance by
making the FR model robust against FAE. To this end, the intuitive solutions are
either to use unaligned faces or to apply random spatial transformations to the
aligned samples. However, it has been shown that random data augmentation is
ineffective in most FR evaluations [36,41,51].
We aim to make the FR network robust to FAE through enriching training
with samples expressing FAE. To this aim, we leverage adversarial data augmentation (detailed in Section 3.2) combined with differentiable spatial transformation _Tθ_ ( _._ ) [28] (composition of scaling, rotation, and translation as detailed
in Section 3.2) to craft adversarial samples expressing FAE [28]. Formally, we
find the _θ_ that crafts hard samples for the FR model, _I.e.,_ maximizing the FR
8 Saadabadi et al.
objective function. Then, the FR network _Fψ_ ( _._ ) is trained using batch formed
from benign and adversarial samples, _I.e.,_ minimization of FR objective. Hence,
our main optimization can be formulated as:
q
: **w** _or_ s _t}_
m
_\_ a _rg_ m **i** _n_ _ { \ps i } _{_ \ _fr_ a _c_ { **1** } _{ |_ \ _athcal {D}|}\sum _{(\mathbf {x},y)\in \mathcal {D}}\left [L(F_\psi ; \mathbf {x}, y) + \argmax _{\theta }L(F_\psi ; T_{\theta }(\mathbf {x}), y)\right ]},_ (7)
\ lab
_e_
l
_{e_
where the maximization is solved using PDG as explained in Equation 2 and _L_
is an arbitrary FR objective, _I.e.,_ ArcFace [11].
_′_
In crafting the adversarial sample **x** = _Tθ∗_ ( **x** ), _I.e.,_ maximization in Equation 7, we do not want **x** _[′]_ to be indistinguishable from **x**, but it should lie on
the manifold of valid training instances. However, the maximization may destroy
all features of the input image. For instance, _Tθ∗_ may zero out the image, making learning infeasible. Typically, the _ℓp_ -norm of the difference between benign
and adversarial samples is employed to define the allowed perturbation set _S_, as
shown in Equation 3. This convention is based on the fact that when two images
are close in a _ℓp_ -norm, they are visually similar [19]. However, its converse does
not always hold, _I.e.,_ two images that are distant in a _ℓp_ -norm, can also be visually similar. For instance, large _ℓ_ 2-norm results from single pixel translation [17].
Thus _ℓp_ -norm constrained _S_ is not applicable here.
Alternatively, we utilize the per-landmark flow (displacement) vector **f** =
_′_ _′_
( _u_ _−_ _Pu_ ( _j_ ) _, v_ _−_ _Pv_ ( _i_ )) [71], _I.e.,_ the vector from a position of a landmark in
_′_
the **x** to its corresponding position in the **x**, to define the _S_ :
e _eq_ **:h** _[t][et]_ [a] _[s]_ p
**l** _{_ **a** _ce_
\ _la b_ e
_}_ _\begin {aligned} \mathcal {S}=\{\theta \:| \: \:\:\sum _{\mathbf {p} \in \mathcal {P}}||\mathbf {f}^{\theta }_{\mathbf {p}}||_{2}\leq \sum _{\mathbf {p} \in \mathcal {P}}\overline {f}_{\mathbf {p}} \}, \end {aligned}_
(8)
**a** _ce_
where _P_ is the set of five landmarks typically employed in FR face alignment [7, 11, 36, 42, 62], and _f_ **p** is the upper bound to the norm of flow vector
corresponding to **p** . Typically, FR training samples are already aligned to a predefined template, _I.e.,_ the positions of landmarks in **x** are already available.
Furthermore, the final landmark location after applying the transformation can
be obtained using Equation 4. Therefore, the proposed constraint does not need
landmark estimation and _S_ can be effectively computed using the FR alignment
template, _I.e.,_ common across available large-scale datasets, and Equation 4.
Section 1 of Supplementary material provides a detailed explanation on _S_ .
Furthermore, to inject uncertainty to the crafted _Tθ_, the PGD step size _α_ is
randomly sampled from the Gaussian distribution _N_ ( _µ, σ_ [2] ):
\ _l_ abe _l_ {fi _n_ a _l_ P _G_ D } \re s _iz_ e _b_   - _x_ _{0_ . _9\_ l _i_ ne **w** i _d t_ h } { !}{$ _\ b_ e _gi n_ { _aligned} \theta _{(k+1)} \gets \proj _{\mathcal {S}} \left ( \theta _{(k)} + \alpha \sgn \left (\nabla _{\theta _{(k)}} L(F_{\psi };T_{\theta _{(k)}}(\bx ), y) \right )\right ); \quad \text {s.t.}\:\: \alpha \sim N(\mu,\sigma ^2), \end {aligned}$}_ (9)
randomized _α_ significantly helps the ARoFace to increase the diversity of crafted
transformations. This diversity is essential to enrich the training with samples
expressing the uncertainty inherent to FAE. Following obtaining the adversarial transformation parameter using Equation 9, the _Fψ_ is trained using a batch
containing benign and adversarial samples, minimization of Equation 7. This
framework enriches the training by spatial transformations that express FAE.
ARoFace for Low-Quality Face Recognition 9
**Table 1:** Comparison with SOTA methods on the IJB-B, IJB-C, and TinyFace when
the backbone is ResNet-100. ‘*’ indicates re-runs with official code due to missing
trained models on the official repository.
|Method|Venue|Train Set|IJB-B
TAR@FAR Identification
1e-6 1e-5 1e-4 Rank1 Rank5|IJB-C
TAR@FAR Identification
1e-6 1e-5 1e-4 Rank1 Rank5|TinyFace
Rank1 Rank5|
|---|---|---|---|---|---|
|MagFace [46]
MagFace+IIC [27]
ArcFace [11]
ArcFace+CFSM* [41]
ArcFace+ARoFace|CVPR2021
ICLR2024
CVPR2019
 ECCV2022|MS1MV2
MS1MV2
MS1MV2
MS1MV2
MS1MV2|42.32 90.36 94.51
-
-
-
-
-
-
-
38.68 88.50 94.09
-
-
47.27 90.52 95.21 95.00
96.67
**48.70 90.83 95.38 95.13 96.71**|89.26 93.67 95.81
-
-
89.38 93.95 95.89
-
-
85.66 92.69 95.74
-
-
** 90.83** 94.72 96.60 96.19
97.30
**   89.28 94.74 96.66 96.20 97.29**|-
-
-
-
-
-
64.69
68.80
**     67.32 72.45**|
|ArcFace+VPL [13]
ArcFace+SC [9]
ArcFace [11]
ArcFace+ARoFace|CVPR2021
ECCV2020
CVPR2019|MS1MV3
MS1MV3
MS1MV3
MS1MV3|-
-
95.56
-
-
-
-
95.25
-
-
40.27 92.09 95.47 95.29
97.01
**42.31 92.85 95.68 95.79 97.44**|-
-
96.76
-
-
-
-
96.61
-
-
90.99 95.31 96.81 96.61
97.66
**   91.48 95.69 96.87 97.07 98.09**|-
-
-
-
63.81
68.05
**     67.54 71.05**|
|ArcFace* [11]
ArcFace+ARoFace|CVPR2019|WebFace4M
WebFace4M|44.67 92.24 95.76** 96.17** 97.51
** 45.10 92.33 95.83** 96.13** 97.62**|91.45 95.43** 97.16** 97.42
98.24
** 91.48 95.69** 96.87** 97.51 98.35**|71.11
74.38
**  73.80 76.53**|
|AdaFace [36]
AdaFace+ARoFace|CVPR2022|WebFace4M
WebFace4M|** 44.48** 92.26 96.03** 96.26** 97.68
 44.21** 93.57 96.35** 96.23** 97.71**|90.43 95.34 97.39 97.52
98.35
** 91.57 95.98 97.51 97.59 98.41**|72.02
74.52
**    73.98 76.47**|
|AdaFace [36]
AdaFace+ARoFace|CVPR2022|WebFace12M
WebFace12M|** 47.49** 93.13** 96.30 96.28** 97.72
 44.76** 93.14** 96.23 96.23** 97.93**|89.47 95.94 97.54 97.56
98.38
** 89.80 96.20 97.60 97.61 98.57**|72.29
74.97
**    74.00 76.87**|
Consequently, it improves the model robustness to FAE which eventually enhances the FR performance on LQ faces. Our proposal is orthogonal to the
FR training objective and improves LQ FR without facing the issue of identity
preservation, and reliance on a _a priori_ defined target dataset. The whole process
is shown in Figure 2 and Algorithm 1.


We utilize the cleaned version of MS-Celeb-1M [21], provided by [11,14] as our
training dataset, _I.e.,_ MS1MV3. This dataset consists of almost 5M images from
90K identities. As per the conventional FR framework, all used datasets in our
work are aligned and transformed to 112 _×_ 112 pixels. We evaluate ARoFace on
TinyFace [6], IJB-B [69], IJB-C [45], and IJB-S [32] datasets.
**TinyFace** [6] is an FR evaluation dataset comprising 169,403 LQ face images
across 5,139 identities, designed for 1: _N_ recognition. The average image size of
this dataset is 20 by 16 pixels.
**IJB-B and IJB-C.** IJB-B [69] contains approximately 21.8K images (11.8K
faces and 10K non-face) and 7k videos (55K frames), representing a total of
1,845 identities. IJB-C [45], an extension of IJB-B, includes 31.3K images and
117.5K frames from 3,531 identities. IJB-B and IJB-C contain both HQ and LQ
samples and have been widely used for evaluating the FR models [36].
**IJB-S** [32] is recognized as one of the most challenging FR benchmarks, primarily utilizing samples from real-world surveillance videos, see Figure 3a. This
dataset comprises 350 surveillance videos, totaling 30 hours, from 202 identities. Additionally, there are seven HQ photos for each subject. This dataset is
characterized by three keywords:
**– Surveillance:** Refers to the use of surveillance video footage.
10 Saadabadi et al.
**Table 2:** Comparison with SOTA methods on the IJB-S benchmark when the backbone
is ResNet-100. ARoFace outperforms all the baselines by a considerable margin.
|Method|Venue|Dataset|Surveillance-to-Single
Rank1 Rank5 1|Surveillance-to-Booking
Rank1 Rank5 1|Surveillance-to-Surveillance
Rank1 Rank5 1|
|---|---|---|---|---|---|
|ArcFace [11]
PFE [53]
URL [55]
ArcFace+ARoFace|CVPR2019
ICCV2019
ICCV2020|MS1MV2
MS1MV2
MS1MV2
MS1MV2|57.35
64.42
41.85
50.16
58.33
31.88
59.79
65.78
41.06
**61.65**
**67.6**
**47.87**|57.36
64.95
41.23
53.60
61.75
35.99
61.98
67.12
42.73
** 60.66 67.33**
**46.34**|-
-
-
9.20
20.82
0.84
-
-
-
**18.31 32.07**
**2.23**|
|ArcFace [11]
ArcFace+ARoFace|CVPR2019|WebFace4M
WebFace4M|69.26
74.31
57.06
** 70.96 75.54 58.67**|70.31
75.15
56.89
**  71.70 75.24**
**58.06**|32.13
46.67
5.32
**32.95 50.30**
**6.81**|
|AdaFace [36]
AdaFace+ARoFace|CVPR2022|WebFace12M
WebFace12M|71.35
76.24
59.40
** 72.28 77.93 61.43**|71.93
76.56
59.37
**  73.01 79.11**
**60.02**|36.71
50.03
4.62
**40.51 50.90**
**6.37**|
**– Single:** Denotes the utilization of a single HQ enrollment image.
**– Booking:** Multiple enrollment images taken from different viewpoints.


We adopt a modified version of ResNet-100 [11] as our backbone. The training is
done for 28 epochs using ArcFace loss, with exceptions noted where applicable.
We employ SGD as the optimizer, with a cosine annealing learning rate starting
from 0.1, a weight decay of 0.0001, and a momentum of 0.9. The component of
_θ_ governing scale transformation is initialized with _N_ ( _µ_ = 1 _, σ_ [2] = 0 _._ 01), while
parameters for rotation and translation are derived from _N_ ( _µ_ = 0 _, σ_ [2] = 0 _._ 01).
Furthermore, _α_ is sampled from _N_ ( _µ_ = 0 _, σ_ [2] = 0 _._ 01). Section 2 of Supplementary
Material provides detailed experiments on these parameters. During training,
each GPU handles a mini-batch of size 512, utilizing four Nvidia RTX 6000
GPUs. For a fair comparison, when a checkpoint for a specific method was not
available, we used the official code released by the authors and the optimal hyperparameters as recommended in their publication for reproducing their results.


Table 1 compares ARoFace performance on IJB-B, IJB-C, and TinyFace against
SOTA methods. These results demonstrate that ARoFace consistently sets new
SOTA benchmarks across a variety of metrics and datasets. These advancements
using different training datasets suggest that ARoFace’s benefits are not confined
to any specific training dataset. When employing the WebFace4M, MS1MV3,
and MS1MV2 training sets, ARoFace outperforms its competitors by 0.43%,
2.04%, and 1.43% at FAR=1 _e −_ 6 on IJB-B, respectively. Concretely, on IJB-C,
ARoFace surpasses AdaFace by 1.14% using WebFace4M and ArcFace by 0.49%
using MS1MV3 at FAR=1 _e −_ 6. Using MS1MV2 as the training data, ARoFace
(ArcFace+ARoFace) outperforms CFSM [41] (ArcFace+CFSM) by 2.63%, and
3.65% improvements in Rank1, and Rank5 TinyFace identification, respectively.
These enhancements over CFSM, which prioritizes the inclusion of synthesized
LQ data in the training, underscore the significance of mitigating the susceptibility to FAE over the introduction of LQ instances to the training.
ARoFace for Low-Quality Face Recognition 11
**Table 3:** Comparison between baseline and baseline+ARoFace on IJB-B and IJB-C
dataset on aligned and alignment-perturbed inputs.
|Method Input Alignment|IJB-B
1e-5 1e-4 1e-3 1e-2|IJB-C
1e-5 1e-4 1e-3 1e-2|
|---|---|---|
|ArcFace [11]
Original
ArcFace+ARoFace
Original
General improvement|92.09
95.47
96.92
97.84
92.85
95.68
96.97
97.88
**+0.76**
**+0.21 +0.05 +0.04**|95.31
96.81
97.88
98.54
95.69
96.87
97.95
98.56
**  +0.38**
**+0.06**
**+0.07 +0.02**|
|ArcFace [11]
Perturbed
ArcFace+ARoFace
Perturbed
Improvement to misalignment|45.74
83.64
88.80
92.34
61.09
90.85
93.17
94.86
**+15.35 +7.21 +4.37 +2.52**|16.17
68.89
85.53
91.11
28.92
84.50
91.40
96.27
**  +12.75 +15.61 +5.87 +5.16**|
|AdaFace [36]
Original
AdaFace+ARoFace
Original
General improvement|90.86
95.84
97.34
98.31
90.05
95.91
97.61
98.34
**-0.81**
**+0.04 +0.27 +0.03**|95.33
97.09
98.14
98.89
95.31
97.12
98.24
98.91
**-0.02**
**+0.03**
**+0.1**
**+0.0.3**|
|AdaFace [36]
Perturbed
AdaFace+ARoFace
Perturbed
Improvement to misalignment|15.71
54.88
84.44
92.46
48.61
82.44
89.40
92.69
**+32.9 +27.56 +4.96 +0.23**|2.16
19.12
57.69
88.14
20.19
64.52
85.23
91.61
**  +18.03 +45.4 +27.54 +3.47**|
Using WebFace4M as the training data, ARoFace boosts AdaFace performance in TinyFace evaluation by 1.82% and 2.11% in Rank1 and Rank5, respectively. Furthermore, using WebFace12M as the training data, ARoFace enhances AdaFace by 2.35%, and 2.5% in Rank1 and Rank5, respectively. These
consistent enhancements demonstrate that ARoFace maintains its effectiveness
as dataset size increases, _I.e.,_ from 4M to 12M. Remarkably, ARoFace achieves
top performance on TinyFace without lowering the results on IJB-B and IJB-C
benchmarks, outperforming its competitors in most verification and identification metrics. IJB-B and IJB-C datasets comprise both HQ and LQ faces [36],
as depicted in Figure 3b. Hence, consistent improvements across IJB-B, IJB-C,
and TinyFace show ARoFace generalization and its ability to enhance LQ FR
without compromising performance on HQ faces.
Furthermore, Table 2 shows ARoFace’s performance on the IJB-S dataset
against its competitors. ARoFace establishes a new SOTA performance by significantly outperforming previous baselines. In particular, ARoFace exceeds the
ArcFace baseline with the MS1MV2 training set, marking improvements of 6.02%
in Surveillance-to-Single and 5.11% in Surveillance-to-Booking verification. This
notable progress not only underscores the face alignment challenges in LQ benchmarks like IJB-S but also demonstrates our method’s ability to improve FR
model robustness against FAE. The consistent improvement across IJB-S metrics highlights the crucial role of FAE in lowering the discriminative power of
existing SOTA FR models, such as ArcFace and AdaFace. These enhancements,
observed with different training datasets, affirm that our method’s effectiveness
is independent of the training set.


Face alignment is an essential prerequisite for almost all available FR models. In
Figure 1, we empirically illustrate the severe vulnerability of SOTA FR networks
to even mild FAE in IJB-B and IJB-C. Here, we further examine the effectiveness of ARoFace in improving model robustness against FAE. Table 3 compares
12 Saadabadi et al.
**Fig. 3:** (a, b) Visualizing samples from IJB-S and IJB-B datasets respectively. IJBB consists of both HQ and LQ instances while IJB-S only consists of LQ probe instances. (c) Visualizing the benign (top : **x** ) and their corresponding adversarial example (bottom : _Tθ_ ( **x** )) produced by ARoFace. (d) Orthogonality of the ARoFace to
different FR objective functions. In all scenarios, integrating ARoFace into training
significantly improved performance on TinyFace.
the performance of ArcFace, and AdaFace under their original training regimes
and our modified training approach. Introducing ARoFace results in considerable improvement in the robustness to FAE for both methods. Specifically, the
ArcFace performance on the perturbed images increases by 15 _._ 35% and 12 _._ 75%
in TAR@FAR=1 _e −_ 5, and 7 _._ 21% and 15 _._ 61% in TAR@FAR=1 _e −_ 4 for IJB-B
and IJB-C, respectively. Consistently, the Adaface performance on the perturbed
images improved by 32 _._ 9% and 18 _._ 03% in TAR@FAR=1 _e −_ 5, and 27 _._ 56% and
45 _._ 4% in TAR@FAR=1 _e −_ 4 for IJB-B and IJB-C, respectively.
These significant improvements showcase the efficacy of the proposed method
in making the FR network more robust against FAE. Note that the inclusion
of classical augmentation techniques in the training of FR models results in
decreasing model generalization [36, 41, 51]. Notably, integrating ARoFace resulted in performance improvement in aligned and alignment perturbed inputs.
This consistent improvement is particularly noteworthy which emphasizes the
generalizability of the proposed method and highlights that ARoFace does not
sacrifice the performance on aligned samples to gain robustness against FAE.

4.4 Analysis on Robustness to Face Alignment Error


ARoFace aims to improve LQ FR by injecting misaligned samples into the training. Figure 3c shows original and augmented samples along each other. Here, we
evaluate the effectiveness of ARoFace across different angular margins, depicted
in Figure 3d. Specifically, we modified the training code of three SOTA methods, _I.e.,_ CosFace, ArcFace, and AdaFace, by incorporating our training policy
and using the CASIA-WebFace [81] as the training data. These results suggest
the orthogonality of ARoFace to existing angular penalty losses. Thus, ARoFace
can be seamlessly integrated into SOTA FR frameworks as a plug-and-play module, increasing their robustness to FAE and ultimately improving performance,
particularly in LQ evaluations.

4.5 Orthogonal Improvement to Angular Margin


One significant benefit of the proposed method over [41, 54] is its ability to
bypass the need for complex image generation procedure, which requires twostep optimization, _I.e.,_ training a generator and then integrating it to the FR
ARoFace for Low-Quality Face Recognition 13
**Fig. 4:** (a, b) Training speed and GPU memory consumption comparison between
CFSM and ARoFace: ARoFace significantly enhances training efficiency and reduces
GPU memory consumption compared to CFSM. (c, d, e) Comparing the evaluation
performance between employing adversarial vs. random spatial transformation during
training: Adversarial improves performance, while random fails on IJB-B and IJB-C.
training. Our proposal crafts an affine transformation [28] to apply directly to
the input image, thereby eliminating the need for two-step optimization.
Moreover, ARoFace employs only four additional trainable parameters, responsible for a negligible increase in computational load. The majority of the
overhead computation time and memory consumption is due to adversarial training. In Figure 4a and b, we compare our method’s training speed and GPU
memory usage with CFSM of Liu _et al_ . [41], which also aims to enhance LQ
FR. By avoiding the usage of image generation models, _I.e.,_ GAN in [41], we
have increased CFSM’s [41] training speed by 25% and decreased the memory
consumption by 35%. These significant improvements in training speed and reduction in memory consumption, coupled with improved performance as shown
in Tables 1 and 2, emphasize the capability of our method for large-scale FR.
Furthermore, we also compare (ArcFace+CFSM+ARoFace) with the original CFSM setup (ArcFace+CFSM). CFSM already employs adversarial optimization. Hence, the integration of ARoFace into CFSM results in a negligible
decrease in training speed and an increase in memory usage, as shown in Figure 4a, b. The extra computational effort from integrating ARoFace is minimal,
confirming that extra computational load stems from the adversarial training
which is necessary for integrating augmentation into FR training [41].


**Table 4:** Ablation on transformation components when the training data is CASIA-WebFace, backbone is ResNet-100 and the objective function is ArcFace.
In Section 3.3, we note that the spatial trans- **Table 4:**
tion components when the train
formation employed in ARoFace comprises
ing data is CASIA-WebFace, back
scaling, rotation, and translation. Here, we
bone is ResNet-100 and the objec
investigate the effect of each component on
tive function is ArcFace.
the performance of ARoFace, _I.e.,_ how much
These experiments, as shown in Table 4, show
that scaling is the most beneficial transfor
|Scale Rotation Translation|Rank1|Rank5|
|---|---|---|
|_×_
_×_
_×_
✓
_×_
_×_
_×_
✓
_×_
_×_
_×_
✓
✓
✓
_×_
✓
✓
✓|54.47
59.70
57.32
57.98
62.32
65.26|59.89
65.17
61.93
62.57
67.01
70.46|
mation, enhancing performance by nearly 5%. This aligns with previous re
14 Saadabadi et al.
search [36, 41, 54], highlighting scaling’s role in generating LQ inputs. Translation and rotation each contribute approximately 3% improvement, reflecting
their similar influence on face alignment.

4.7 Ablation on Transformation Components