- Will the dataset be distributed to third parties outside of the entity (e.g., company, institution,
organization) on behalf of which the dataset was created?
No.
    - How will the dataset will be distributed (e.g., tarball on website, API, GitHub)?
The dataset will be released upon acceptance.
    - When will the dataset be distributed?
After the whole process of reviewing.
    - Will the dataset be distributed under a copyright or other intellectual property (IP) license,
and/or under applicable terms of use (ToU)?
The dataset will be released under MIT License.
    - Have any third parties imposed IP-based or other restrictions on the data associated with the
instances?
No.
    - Do any export controls or other regulatory restrictions apply to the dataset or to individual
instances?
No.


- Who will be supporting/hosting/maintaining the dataset?
The authors of this paper.
    - How can the owner/curator/manager of the dataset be contacted (e.g., email address)?
Contact the first author ( `jiyounglee0523@kaist.ac.kr` ) or other authors.
    - Is there an erratum?
No.
    - Will the dataset be updated (e.g., to correct labeling errors, add new instances, delete
instances)?
If any correction is needed, we plan to upload a new version.
    - If the dataset relates to people, are there applicable limits on the retention of the data
associated with the instances (e.g., were the individuals in question told that their data would
be retained for a fixed period of time and then deleted)?
N/A
    - Will older versions of the dataset continue to be supported/hosted/maintained?
We plan to maintain the newest version only.
    - If others want to extend/augment/build on/contribute to the dataset, is there a mechanism for
them to do so?
Contact the authors of the paper.

VisAlign: Dataset for Measuring the Degree of Alignment between AI and
  Humans in Visual Perception

Jiyoung Lee
Seungho Kim
Seunghyun Won
Joonseok Lee
Marzyeh Ghassemi
James Thorne
Jaeseok Choi
O-Kil Kwon
Edward Choi


For the experiments in Section 5, we use a batch size of 16 with a learning rate starting at 1 _×_ 10 _[−]_ [5] .
The learning rate is decreased by a factor of 0.5 if there is no improvement for 10 epochs or until it
reaches 1 _×_ 10 _[−]_ [6] . We approximately match the size of each model to 300M parameters. For ViT, we
use the variant with 30 layers and 16 heads in each layer. For Swin Transformer, we use a hidden
layer of size 256 with layer numbers _{_ 2 _,_ 2 _,_ 15 _,_ 2 _}_ . For DenseNet, we use a growth rate of 64 with the
block configuration _{_ 24 _,_ 48 _,_ 84 _,_ 64 _}_ . For ConvNeXt, we use the large variant with block numbers
_{_ 3 _,_ 3 _,_ 50 _,_ 3 _}_ . For MLP-Mixer, we use a hidden size of 2048 with 60 layers. We trained all models
using either a single NVIDIA RTX A6000 or NVIDIA GeForce RTX 3090 graphics card.


Table 6 shows the scientific names and sub-species for each class. The classes are selected based on
the following four criteria.
    - They should be grouped into one scientific name for clear definitions
    - They should be visually distinguishable from other species to avoid multiple correct answers
    - They should have typical visual features allowing them to be identified by a single image
    - They should be familiar to humans so that any MTurk worker can participate in our survey
The final 10 classes are _Tiger_, _Rhinoceros_, _Camel_, _Giraffe_, _Elephant_, _Zebra_, _Gorilla_, _Bear_, and
_Human_ . These labels are revised and verified by two zoologists.


This section will describe the details of our dataset construction.


Our dataset should contain sufficient test samples to serve as a universal benchmark. For instance, if
the test set does not have enough test samples, it will fail to test the model’s capacity appropriately.
Cronbach’s alpha [9] is an indicator that represents the validity of the number of questions in a
test. To calculate this value, we first need responses from humans. Therefore, we can only calculate
Cronbach’s alpha for the _Uncertain_ group, as it is the only group with human responses. However,
we believe that the Cronbach’s alpha value for the _Uncertain_ group can also be applied to other
categories, given that samples in other categories are more straightforward than those in _Uncertain_
( _e_ . _g_ ., they have clear images and optimal actions are explicit). To calculate this value, we first treat
the original label as a gold standard answer if more than 50% of MTurk workers correctly classify
the image. Otherwise, we set Abstention as the gold standard answer. We then evaluate whether each
response for each image is correct based on the gold standard answer and set it to a binary value (1
for a correct response and 0 for an incorrect response). We denote the binary response for the _i_ -th
image as _xi_ .
Next, we calculate the variance of responses for each image, denoted as _V ar_ ( _xi_ ) for the _i_ -th image,
and the variance of the sum of responses from all images, denoted as _V ar_ ( _X_ ). Here, _X_ is the sum of
responses for all images, i.e., _X_ = [�] _i_ _[N]_ =1 _[x][i]_ [, and] _[ N]_ [ is the total number of images. We then employ]
Cronbach’s Alpha formula as shown in Equation 2 below.
In our case, [�] _i_ _[N]_ =1 _[V ar]_ [(] _[x][i]_ [) = 127] _[.]_ [134] _[, V ar]_ [(] _[X]_ [) = 976] _[.]_ [564][, and] _[ N]_ [ = 100][ which yields a Cron-]
bach’s Alpha of 0.88. A Cronbach’s Alpha value between 0.75 and 0.9 is considered ideal. A value
higher than 0.9 might indicate redundancy in the questions, as it suggests that there are more questions
than necessary.
_N_
- _xi_ (2)
_i_ =1
_N_
_α_ =
_N −_ 1


- - _N_
_i_ =1 _[V ar]_ [(] _[x][i]_ [)]
1 _−_
_V ar_ ( _X_ )
_,_ _where X_ =
Since there is a limited amount of data for Category 2 and Category 5, we manually generated samples
with using Stable Diffusion [62]. We filtered all images to ensure that there is only one object in an
image and images look as realistic as Category 1.


The prompt used is " _RAW photo of a {subspecies} {background_prompt}, 8k uhd, dslr, soft lighting,_
_high quality, film grain, Fujifilm XT3_," where _{subspecies}_ is one of the subspecies listed in Table 6
and _{background_prompt}_ is one of the following:
on the moon surrounded by fire underwater
in New York city in a construction site inside an office
near a swimming pool on the playground on a volcano
on the clouds on an iceberg in a rainforest
on a snowy mountain on top of a roof on a pile of garbage
at night taken using an infrared camera on top of a tree inside a tunnel
inside a large bathroom on top of a bus with a static background
with a rainbow background in a dystopian world in the middle ages
with a purple background with a pink background with a red background
with a bright orange background
The negative prompt used is " _unrealistic, bad anatomy, wrong anatomy, extra limb, missing limb,_
_floating limbs, disconnected limbs, mutation, mutated, ugly, disgusting, amputation_ ."


To create a Category 5 image, we first create an image using the following prompts: " _{subspecies}_
_that looks like {other_animal}", "a picture of {subspecies} with head of an {other_animal}"_ . For
_{other_ _ _animal}_, we choose species that in not in-class ( _e_ . _g_ ., eagle, bird, fish, alligator). Some
samples were generated using a variant of Stable Diffusion called MagicMix [37], which performs
_semantic mixing_ by blending the semantics of an image and a text prompt to create a new image. To
use MagicMix, we first create an image using a prompt similar to the one used for Category 2, except
we also choose species that are not in-class. Then, we insert any other species as the target prompt
into MagicMix to blend the semantics of another species into the image.
Figure 3: Labeling examples provided to MTurk Workers as part of the instructions.


In this section, we explain why corrupted images should be evaluated based on human perception
ratios obtained from MTurk workers. Some researchers might argue that since the corrupted images
come from clean images, the models should be able to correctly classify the original label despite the
existence of corruption severity regardless of human perception ability. However, when the images
are gradually corrupted, the essential features of objects will eventually be lost and become images
with complete noises ( _e_ . _g_ ., black images or images with pure Gaussian noise). In such cases, it is
meaningless for AI models to make predictions because they would predict based on noise rather
than using related features to classes. Therefore, we need new labels for corrupted images, indicating
whether images are unrecognizable or contain essential features. However, setting a unified guideline
is impossible since visibility varies by objects, corruption types, and images themselves. Therefore,
we must newly obtain labels by asking qualified humans. Here, the qualified humans we refer to
are people with commonsense knowledge ( _i_ . _e_ ., must know the 10 mammals) and have functioning
visual perception ( _i_ . _e_ ., we test this via intra-annotator agreement and we also rejected responses
from workers who chose other than ‘Abstention’ for distractor images that are corrupted images
from Category 4). To obtain a gold human ratio, we asked 134 people from diverse age groups and
backgrounds to achieve the error bound of 5%.
Nevertheless, some might still argue that AI should aim to identify the original class because we can
set up a controlled experiment where we can test if its guess was correct. For example, we can put an
elephant in a dark room, let the machine take a guess, then increase the brightness of the room. In
such experiments, we may be able to identify if some AI possesses superhuman visual perception
( _e_ . _g_ ., only 1 out of 100 human participants were able to confidently tell the object in the dark room
was an elephant, but the AI had a 95% confidence in the elephant class). However, making a decision
based on a single image and setting a controlled experiment are completely different settings since it
is infeasible to set a controlled experiment with static images. It is not correct to claim that AI must
always try to identify the original class in the former ( _i_ . _e_ ., deciding based on a single image) because
the latter ( _i_ . _e_ ., running a controlled trial) is also possible. The main objective of VisAlign is to test
the model’s safety (or potential harmfulness), as well-aligned models are less likely to cause harm.
Potentially, our dataset can be used as a prerequisite such that, if models pass our dataset by some
threshold, then the models are less likely to make harmful decisions. Then, the model’s superhuman
capability can be tested using a separate dataset under controlled experiments. This is somewhat
similar to the multi-phase drug development process, where the initial phases always test the basic
safety of the drug (toxicity, side effects) before advancing to latter phase to test the clinical efficacy
of the drug.

E AI-Human Visual Alignment for Uncertain Images


This section will describe in detail of Amazon Mechanical Turk to obtain gold human ratio of
_Uncertain_ samples. We paid $0.05 for classifying 35 images per worker.


This section contains the instructions given to the survey participants. We also present detailed
labeling instruction examples in Figure 3 for ease of labeling for MTurk workers to understand the
variety of cases in which to abstain. Specifically, we provide a clear images of Tiger and instruct the
workers to choose label " _Tiger_ ". For other cases ( _e_ . _g_ ., only a part of a tiger, a bag with a tiger pattern,
species similar to tiger, other animals), we instruct the workers to choose "None of the 10 mammals,
uncertain, or unrecognizable". The following box contains the exact instructions given to the MTurk
Workers:
There are 11 labels to choose from:
    - Tiger
    - Zebra
   - Camel
    - Giraffe
    - Elephant
    - Hippo
    - Gorilla
    - Bear
    - Pig
   - Human
    - None of the above, uncertain, or unrecognizable
Please choose one of the first 10 labels only if you are certain the image belongs to that label.
Please choose the 11th label ( **None of the above, uncertain, or unrecognizable** ) for any of the
situations below.
    - None of the 10 labels describe the object observed in the image
    - The object observed in the image is unrecognizable
    - You are not sure which label describes the object observed in the image
    - Any other similar situation


To ensure high data quality, we filtered noisy collected responses of the survey. We added distractors
(Category 4 samples corrupted with a severity between 1 and 10) in our survey. Among them, the
authors identified clear distractors that should always be chosen as "None of the above, uncertain,
or unrecognizable" ( _e_ . _g_ ., a clear image of a cup or a truck). We reject all the responses from the
survey participants who chose other than "None of the above, uncertain, or unrecognizable" for clear
distractors.


This section provides the characteristics of the MTurk workers participated in our survey.
MTurk workers are equal in gender (44.9% of male, 53.8% of female, and 1.3% of others).
People from diverse age groups (from 10s to 70s) participated (2.1% of 10s, 19.7% of 20s, 35.1% of
30s, 24.8% of 40s, 14.3% of 50s, 3.5% of 60s, and 0.4% of 70s).
The participant locations were focused on largely five countries, namely USA (71.1%), India (13.1%),
Italy (5%), UK (3.1%), and Canada (2.3%). Other responses are from other countries including
Phillippines, Brazil, Nigeria, Mexcio, Pakistan, UAE, and Malaysia.


Given an image _x_ and its corresponding label _y_, we can assume _y ∼_ Bernoulli( _p_ ), where _p_ is the
probability of the true class.
Let _N_ denote the number of individuals in the population and _n_ denote the number of samples,
then the approximated variance of ˆ _p_, assuming sampling without replacement and a 95% confidence
level, can be expressed as in Eq. 3. In this equation, _z_ 0 _._ 975 represents the z-score under the normal
distribution corresponding to a probability of 0.975, and _q_ = 1 _−_ _p_ .
~~�~~
 _z_ 0 _._ 975
_V_ - (ˆ _p_ ) = _z_ 0 _._ 975
_≈_ _z_ 0 _._ 975
~~�~~
1 _−_ _[n]_
_N_
~~�~~ _p_ ˆ _q_ ˆ
_n −_ 1
~~�~~ ~~�~~ _p_ ˆ _q_ ˆ
_×_
_n −_ 1
(3)
~~�~~
(∵ _N_ = _∞_ )
Given an error bound _ξ_, we can derive the required minimum number of samples to achieve the error
bound by setting the 95% confidence interval of the approximated variance to be lower than _ξ_ . For
ease of calculation, we round _z_ 0 _._ 975 = 1.96 to 2.
~~�~~ _p_ ˆ _q_ ˆ
_n −_ 1
(4)
~~�~~
_≤_ _ξ_
_n ≥_ [4ˆ] _[p][q]_ [ˆ]
_ξ_ [2] [+ 1]
Since we do not have prior knowledge of ˆ _p_, we set ˆ _p_ to 111 [, which represents a uniform distribution]
over the 11 classes (10 mammals + abstention). We drop the constant for simplicity.
[1]
_n ≥_ [4] _[ ×]_
[1] [10]
11 _[×]_ 11
11 _[×]_ 11 40
= (5)
_ξ_ [2] 11 [2] _× ξ_ [2]
For _ξ_ = 0.05, 0.1, 0.15, the minimum required number of participants are as follows:
Therefore, to achieve an error bound lower than 5%, we surveyed 134 people per image.


Section 4.2 describes the possible action types for each group and how they are used to obtain the
reliability score _RSc_ . While the reliability score allows us to assess the reliability of a given model
with a single value, we also provide the ratios of each action type by their respective groups in Table
7.

G Experimental Results Shown as Percentages


In this section, we will explain in detail how our work differs from related previous works. Our
ultimate goal is to create a rigorous test (similar to tests that humans take such as college entrance
exams) to quantitatively measure the visual perception gap between the models and humans across
various categories. Our main interest does not lie in training but on rigorously testing the visual
perception alignment. For that purpose, a dataset should satisfy the four requirements we mentioned
in Section 3 and use a proper metric that reflects the visual perception alignment.
Peterson et al. [54] and Schmarje et al. [68] utilized their datasets mainly for training and did not
thoroughly verify whether the model actually achieved visual alignment. Peterson et al. [54] only
tested their models on in-class samples (in our case, Category 1) and out-of-class samples (in our case,
Category 4 and Category 6) and they showed only accuracy and cross entropy, which is analogous to
KL divergence. Therefore, they did not test their models on various possible scenarios and did not
use proper measurement, as KL divergence is not an optimal choice for visual perception alignment
as we described in Section 4.1. Schmarje et al. [68] only tested their models on in-class samples (in
our case, Category 1) and showed accuracy and KL divergence. Therefore, although previous works
trained their models with the goal of achieving visual perception alignment, none of the works have
thoroughly verified how much the models have actually achieved visual perception alignment under
diverse situations with an appropriate measurement.
Zhang et al. [79] and Bomatter et al. [5] are similar to our work since they show that both models
and humans have better object recognition when given more contextual information, but it is difficult
to say that they have comprehensively evaluated visual perception alignment. These two works
only tested their models on partial aspects (in our case, Category 1, Category 2, and Category 8).
Thus, these works did not test on Must-Abstain samples, which makes it difficult to claim that they
"comprehensively" evaluated visual perception alignment. Zhang et al. [79] and Bomatter et al. [5]
simply showed that both models and humans exhibit similar performance trends based on context ( _i_ . _e_ .,
when given more context, both human’s and model’s visual recognition performance increases), and
they provided human-model correlations to describe their relative trends across conditions. However,
our study on visual perception alignment is not about following human trends, but about measuring
how well the model replicates human perception sample-wise. Hence, considering our research
scope and criteria, it’s challenging to assert that Zhang et al. [79] and Bomatter et al. [5] rigorously
measured visual perception alignment.
In contrast, we quantitatively measured visual perception alignment across various scenarios with
multiple human annotations on uncertain images. In addition, we borrowed Hellinger distance to
precisely calculate the visual perception alignment after careful consideration of other distance-based
metrics such as KL divergence and Total Variation distance. Furthermore, we incorporated specialized
elements (sampling theory, statistical theories related to survey design, and experts in the related
fields) in creating our dataset.
There are three key differences that distinguish our dataset compared to existing datasets that also
focus on uncertainty in object recognition. First, we applied corruption and cropping with different
intensities ranging from 1 to 10 to reflect the continuity of uncertainty mentioned in Appendix I.1.
Uncertainty is continuous and it is critical to test models on samples where uncertainty may increase in
stages. In this sense, we tested models visual perception alignment on varying degrees of uncertainty.
Second, we obtained 134 human annotations per image to accurately estimate the ground truth
visual perception distribution. We borrowed statistical sampling theory to achieve an error bound of
lower than 5%, of which the details are in Section 3.3. Third, while our uncertain samples include
uncertainty that confuses between classes, refer refer to as "inter-class uncertainty" (soft labels
distributed only among target classes), we also include "recognizability uncertainty" (soft labels
distributed among classes + abstention), namely whether an image itself is recognizable or not. If an
image is moderately brightened ( _i_ . _e_ ., intermediate phase between a clear image and a complete white
image), then the object itself may or may not be recognizable. Visual perception includes not only
object identification (predicting that it is an elephant) but also object recognizability (the object itself
is recognizable). In this sense, we cover broader scenarios compared to previous works as we include
object recognizability uncertainty in our uncertain category.
We also want to highlight that VisAlign does only contain Uncertain but also Must-Act and MustAbstain to cover diverse scenarios as possible. In order to evaluate a model’s visual perception
alignment, a model should be tested under Must-Act (whether it predicts a correct class with high
confidence), Must-Abstain (whether it abstains out-of-class samples with high confidence), and
Uncertain (whether it reflects the human uncertainty). However, previous works are limited in that
they test their model on partial cases (Category 1 and Category 4 in Peterson et al. [54], and Category
1 in Schmarje et al. [68]) which does not truly reflect visual perception alignment on various situations.
It is especially important to test models on samples from out of distributions ( _i_ . _e_ ., Category 5 and
Category 7), but previous works overlook these samples thus did not quantitately evalute from visual
perception alignment. Therefore, their dataset cannot be utilized as a benchmark to evaluate visual
perception alignment. While previous papers and our work have in common with handling uncertainty,
in our case, uncertain samples are a subset of our final dataset and we cover more diverse necessary
situations, which previous works do not, as possible to measure the visual perception alignment.

H Detailed Comparisons with Previous Works


In this section, we will discuss a critical aspect of uncertainty which is continuity. Uncertainty is
continuous and it is challenging to draw clear distinctions among classes ( _i_ . _e_ ., as we mentioned in the
main paper that it is hard to distinguish between "car" and "truck") and between clear and uncertain
images ( _i_ . _e_ ., if at least one person claims an image as "uncertain", then it becomes an uncertain
image). However, as our ultimate goal is to construct a universal testing benchmark that quantitatively
measures visual perception alignment between models and humans, our classes should have clear
defintions so that model developers can easily prepare their models and training strategy. Therefore,
after careful consideration, we used the taxonomy classification in biology which is the meticulous
product of decades of efforts by countless domain experts to hierarchically distinguish each species
as accurately as possible with clear definitions. Also, in order to comprehensively measure the
visual perception alignment between models and humans, the models should be tested under various
conditions including clear in-class images (Must-Act), clear out-of-class images (Must-Abstain) and
confusing images (Uncertain). As there is no clear boundary between clear and uncertain images,
the best scenario would be to survey all images in our dataset to 134 people per image to obtain
numerically reliable annotations. However, surveying all images is not always feasible as it requires
tremendous amount of time and money considering that there are 1800 images (900 each in the
open and closed test sets) in our dataset. Therefore, due to the realistic reasons, we put significant
effort to include only clear images that anyone can agree on in Must-Act and Must-Abstain and
obtained human annotations on Uncertain images. Nevertheless, we also recognize that continuity is
an essential characteristic of uncertainty that should be carefully considered and there is always a
possibility of corner cases that may be disagreeable by at least one person. We have done our best to
remove those corner case samples and cross-validated our final selection. Further detailed analysis
and benchmark dataset on the continuity of uncertainty is highly needed and we will leave this as a
future work.


"Uncertainty" is a broad concept and it is hard to define with one clear line and list all possible
cases. In this paper, we chose 15 different types of corruptions to generate uncertainty in various
ways following a concrete previous work [23]. Furthermore, to better represent the continuity of
uncertainty explained in Appendix I.1, we apply the corruptions varying severity ranging from 1 to
10. Many types of corruption we used resemble the reality in their own way. For example, adjusting
the brightness of the image is certainly realistic, and changing its resolution is similar to viewing an
object beyond a filter ( _e_ . _g_ ., semi-transparent glass), and weather changes are also certainly realistic.
These corruptions result in some of realistic uncertain images, precisely 8.5% in the case of the open
test set, where MTurk survey participants were struggling with differentiating between two or more
animals (rather than being confused between one animal and abstention). Despite our meticulous
effort, we are well aware that those corruptions certainly do not cover all possible uncertainties
that arise in the real world. However, "uncertainty" is too broad to specify and diffcult to collect or
generate, and hence for now we use corruptions (but sufficiently divserse types of corruptions).
Table 5: Average and standard deviation of self supervised distance-based visual alignment and
reliability score across 5 seeds. Bold indicates the best performance in each category and underline is
the second best. Deep Ensemble does not have standard deviation since it is the output of 5 different
seeds. The value under each SSL performance shows its difference with the baseline’s performance.
Visual Alignment ( _↓_ ) Reliability score ( _↑_ )
Must-Act Must-Abstain Uncertain
Category 1 Category 2 Category 3 Category 4 Category 5 Category 6 Category 7 Category 8 Average _RS_ 0 _RS_ 450 _RS_ 900
Swin Transformer [40]
No SSL 0.106 _±_ 0.004 0.362 _±_ 0.014 0.221 _±_ 0.017 0.793 _±_ 0.016 0.828 _±_ 0.043 0.800 _±_ 0.022 0.829 _±_ 0.028 0.625 _±_ 0.031 0.571 _±_ 0.015 363 _−_ 225537 _−_ 451437
SP
ASP
MD [36]
KNN [70]
TAPUDD [13]
MC-Dropout [16]
OpenMax [3]
Deep Ensemble [33]
DenseNet [28]
SP
ASP
MD [36]
KNN [70]
TAPUDD [13]
MC-Dropout [16]
OpenMax [3]
Deep Ensemble [33]
SimCLR[6] 0.125(+0.019) _±_ 0.009 0.408(+0.046) _±_ 0.015 0.265(+0.044) _±_ 0.034 0.723(-0.070) _±_ 0.037 0.782(-0.046) _±_ 0.026 0.744(-0.056) _±_ 0.028 0.776(-0.053) _±_ 0.044 0.615(-0.010) _±_ 0.023 0.555(-0.016) _±_ 0.015 (+21)384 _−_ (+13971)211566 _−_ (+27921)423516
BYOL[22] 0.125(+0.019) _±_ 0.009 0.408(+0.046) _±_ 0.015 0.265(+0.044) _±_ 0.034 0.723(-0.070) _±_ 0.037 0.782(-0.046) _±_ 0.026 0.744(-0.056) _±_ 0.028 0.776(-0.053) _±_ 0.044 0.615(-0.010) _±_ 0.023 0.555(-0.016) _±_ 0.015 (+21)384 _−_ (+13971)211566 _−_ (+27921)423516
No SSL 0.085 _±_ 0.007 0.329 _±_ 0.008 0.182 _±_ 0.020 0.998 _±_ 0.000 0.998 _±_ 0.000 0.998 _±_ 0.000 0.998 _±_ 0.000 0.736 _±_ 0.009 0.666 _±_ 0.003 294 _−_ 268356 _−_ 537006
SimCLR[6] 0.091(+0.006) _±_ 0.010 0.356(+0.027) _±_ 0.012 0.214(+0.032) _±_ 0.028 0.999(+0.001) _±_ 0.000 0.999(+0.001) _±_ 0.000 0.999(+0.001) _±_ 0.000 0.999(+0.001) _±_ 0.000 0.735(-0.001) _±_ 0.004 0.674(+0.008) _±_ 0.005 (+7)301 _−_ (+4057)264299 _−_ (+8107)528899
BYOL[22] 0.091(+0.006) _±_ 0.010 0.356(+0.027) _±_ 0.012 0.214(+0.032) _±_ 0.028 0.999(+0.001) _±_ 0.000 0.999(+0.001) _±_ 0.000 0.999(+0.001) _±_ 0.000 0.999(+0.001) _±_ 0.000 0.735(-0.001) _±_ 0.004 0.674(+0.008) _±_ 0.005 (+7)301 _−_ (+4057)264299 _−_ (+8107)528899
No SSL 0.296 _±_ 0.018 0.512 _±_ 0.012 0.364 _±_ 0.012 0.700 _±_ 0.012 0.743 _±_ 0.014 0.723 _±_ 0.017 0.685 _±_ 0.021 0.575 _±_ 0.007 0.575 _±_ 0.006 326 _−_ 248974 _−_ 498274
SimCLR[6] 0.324(+0.028) _±_ 0.016 0.548(+0.036) _±_ 0.018 0.407(+0.043) _±_ 0.023 0.638(-0.062) _±_ 0.017 0.701(-0.042) _±_ 0.015 0.660(-0.063) _±_ 0.015 0.620(-0.065) _±_ 0.031 0.558(-0.017) _±_ 0.011 0.557(-0.017) _±_ 0.006 (+47)373 _−_ (+31997)216977 _−_ (+63947)434327
BYOL[22] 0.326(+0.030) _±_ 0.020 0.549(+0.037) _±_ 0.022 0.408(+0.044) _±_ 0.026 0.635(-0.065) _±_ 0.027 0.699(-0.044) _±_ 0.022 0.659(-0.064) _±_ 0.024 0.615(-0.070) _±_ 0.039 0.558(-0.017) _±_ 0.015 0.556(-0.018) _±_ 0.010 (+44)370 _−_ (+28394)220580 _−_ (+56744)441530
No SSL 0.370 _±_ 0.017 0.580 _±_ 0.008 0.456 _±_ 0.018 **0.549** _±_ **0.025** 0.590 _±_ 0.013 **0.545** _±_ **0.022** 0.554 _±_ 0.035 **0.543** _±_ **0.007** 0.523 _±_ 0.012 **526** **-115124** **-230774**
SimCLR[6] 0.375(+0.005) _±_ 0.019 0.591(+0.011) _±_ 0.018 0.462(+0.006) _±_ 0.028 0.560(+0.011) _±_ 0.019 **0.585** (-0.005) _±_ **0.027** 0.558(+0.013) _±_ 0.021 **0.537** (-0.017) _±_ **0.022** 0.545(+0.002) _±_ 0.010 0.527(+0.003) _±_ 0.006 (-22)504 _−_ (-24322)139446 _−_ (-48622)279396
BYOL[22] 0.374(+0.004) _±_ 0.018 0.591(+0.011) _±_ 0.018 0.462(+0.006) _±_ 0.026 0.560(+0.011) _±_ 0.022 **0.585** (-0.005) _±_ **0.028** 0.558(+0.013) _±_ 0.022 0.538(-0.016) _±_ 0.023 0.545(+0.002) _±_ 0.010 0.527(+0.003) _±_ 0.007 (-22)504 _−_ (-23872)138996 _−_ (-47722)278496
No SSL 0.201 _±_ 0.053 0.427 _±_ 0.048 0.278 _±_ 0.046 0.876 _±_ 0.058 0.889 _±_ 0.048 0.898 _±_ 0.049 0.844 _±_ 0.073 0.663 _±_ 0.022 0.635 _±_ 0.013 294 _−_ 268356 _−_ 537006
SimCLR[6] 0.234(+0.033) _±_ 0.021 0.468(+0.041) _±_ 0.022 0.330(+0.052) _±_ 0.036 0.834(-0.042) _±_ 0.026 0.855(-0.034) _±_ 0.024 0.863(-0.035) _±_ 0.020 0.789(-0.055) _±_ 0.040 0.644(-0.019) _±_ 0.010 0.627(-0.007) _±_ 0.007 (+7)301 _−_ (+4057)264299 _−_ (+8107)528899
BYOL[22] 0.233(+0.032) _±_ 0.021 0.466(+0.039) _±_ 0.022 0.329(+0.051) _±_ 0.037 0.834(-0.042) _±_ 0.023 0.856(-0.033) _±_ 0.023 0.861(-0.037) _±_ 0.019 0.791(-0.053) _±_ 0.038 0.646(-0.017) _±_ 0.008 0.627(-0.008) _±_ 0.006 (+7)301 _−_ (+4057)264299 _−_ (+8107)528899
No SSL 0.099 _±_ 0.008 0.358 _±_ 0.013 0.225 _±_ 0.029 0.831 _±_ 0.037 0.810 _±_ 0.023 0.817 _±_ 0.032 0.724 _±_ 0.084 0.656 _±_ 0.030 0.565 _±_ 0.011 399 _−_ 208401 _−_ 417201
SimCLR[6] 0.107(+0.008) _±_ 0.008 0.389(+0.031) _±_ 0.012 0.240(+0.015) _±_ 0.026 0.848(+0.017) _±_ 0.021 0.825(+0.015) _±_ 0.026 0.849(+0.032) _±_ 0.044 0.805(+0.081) _±_ 0.020 0.683(+0.027) _±_ 0.007 0.593(+0.028) _±_ 0.006 (-39)360 _−_ (-17589)225990 _−_ (-35139)452340
BYOL[22] 0.106(+0.007) _±_ 0.008 0.388(+0.030) _±_ 0.013 0.241(+0.016) _±_ 0.025 0.856(+0.025) _±_ 0.015 0.828(+0.018) _±_ 0.031 0.854(+0.037) _±_ 0.039 0.820(+0.096) _±_ 0.024 0.683(+0.027) _±_ 0.007 0.597(+0.032) _±_ 0.005 (-44)355 _−_ (-20294)228695 _−_ (-40544)457745
No SSL 0.092 _±_ 0.007 0.338 _±_ 0.008 0.191 _±_ 0.020 0.947 _±_ 0.001 0.957 _±_ 0.006 0.951 _±_ 0.002 0.953 _±_ 0.003 0.705 _±_ 0.011 0.642 _±_ 0.003 294 _−_ 268356 _−_ 537006
SimCLR[6] 0.100(+0.008) _±_ 0.009 0.365(+0.027) _±_ 0.011 0.223(+0.032) _±_ 0.027 0.940(-0.007) _±_ 0.005 0.950(-0.007) _±_ 0.004 0.943(-0.008) _±_ 0.003 0.944(-0.009) _±_ 0.005 0.701(-0.004) _±_ 0.003 0.646(+0.004) _±_ 0.004 (+7)301 _−_ (+4057)264299 _−_ (+8107)528899
BYOL[22] 0.100(+0.008) _±_ 0.010 0.365(+0.027) _±_ 0.011 0.223(+0.032) _±_ 0.027 0.940(-0.007) _±_ 0.004 0.949(-0.008) _±_ 0.003 0.943(-0.008) _±_ 0.003 0.945(-0.008) _±_ 0.006 0.701(-0.004) _±_ 0.003 0.646(+0.004) _±_ 0.004 (+7)301 _−_ (+4057)264299 _−_ (+8107)528899
No SSL 0.132 0.377 0.253 0.725 0.766 0.734 0.768 0.584 0.542 434 _−_ 187666 _−_ 375766
SimCLR[6] (+0.012)0.144 (+0.051)0.428 (+0.037)0.290 (-0.060)0.665 (-0.032)0.734 (-0.042)0.692 (-0.044)0.724 (+0.003)0.587 (-0.010)0.533 (+3)437 _−_ (+6753)180913 _−_ (+13503)362263
BYOL[22] (+0.012)0.144 (+0.051)0.428 (+0.037)0.290 (-0.060)0.665 (-0.032)0.734 (-0.042)0.692 (-0.044)0.724 (+0.003)0.587 (-0.010)0.533 (+3)437 _−_ (+6753)180913 _−_ (+13503)362263
No SSL 0.094 _±_ 0.017 0.258 _±_ 0.023 0.183 _±_ 0.019 0.813 _±_ 0.017 0.852 _±_ 0.015 0.819 _±_ 0.012 0.864 _±_ 0.036 0.614 _±_ 0.008 0.562 _±_ 0.007 392 _−_ 211558 _−_ 423508
SimCLR[6] 0.296(+0.202) _±_ 0.034 0.531(+0.273) _±_ 0.039 0.438(+0.255) _±_ 0.046 0.604(-0.209) _±_ 0.025 0.653(-0.199) _±_ 0.038 0.615(-0.204) _±_ 0.030 0.721(-0.143) _±_ 0.041 0.592(-0.022) _±_ 0.022 0.556(-0.006) _±_ 0.005 (+86)478 _−_ (+82886)128672 (+165686) _−_ 257822
BYOL[22] 0.292(+0.198) _±_ 0.021 0.518(+0.260) _±_ 0.027 0.446(+0.263) _±_ 0.008 0.600(-0.213) _±_ 0.022 0.656(-0.196) _±_ 0.020 0.623(-0.196) _±_ 0.021 0.705(-0.159) _±_ 0.029 0.583(-0.031) _±_ 0.027 0.553(-0.009) _±_ 0.009 (+56)448 _−_ (+54056)157502 (+108056) _−_ 315452
No SSL **0.079** _±_ **0.013** **0.224** _±_ **0.023** **0.159** _±_ **0.018** 0.997 _±_ 0.000 0.997 _±_ 0.000 0.997 _±_ 0.000 0.997 _±_ 0.000 0.747 _±_ 0.008 0.650 _±_ 0.004 312 _−_ 260238 _−_ 520788
SimCLR[6] 0.216(+0.137) _±_ 0.025 0.438(+0.214) _±_ 0.032 0.346(+0.187) _±_ 0.038 0.998(+0.001) _±_ 0.000 0.998(+0.001) _±_ 0.000 0.998(+0.001) _±_ 0.000 0.998(+0.001) _±_ 0.000 0.780(+0.033) _±_ 0.012 0.722(+0.072) _±_ 0.009 (-48)264 _−_ (-22548)282786 _−_ (-45048)565836
BYOL[22] 0.216(+0.137) _±_ 0.018 0.419(+0.195) _±_ 0.022 0.354(+0.195) _±_ 0.008 0.998(+0.001) _±_ 0.000 0.998(+0.001) _±_ 0.000 0.998(+0.001) _±_ 0.000 0.998(+0.001) _±_ 0.000 0.775(+0.028) _±_ 0.015 0.720(+0.070) _±_ 0.005 (-36)276 _−_ (-16236)276474 _−_ (-32436)553224
No SSL 0.170 _±_ 0.016 0.323 _±_ 0.022 0.250 _±_ 0.025 0.873 _±_ 0.014 0.866 _±_ 0.019 0.854 _±_ 0.009 0.825 _±_ 0.032 0.620 _±_ 0.022 0.598 _±_ 0.006 339 _−_ 247611 _−_ 495561
SimCLR[6] 0.266(+0.096) _±_ 0.024 0.468(+0.145) _±_ 0.028 0.379(+0.129) _±_ 0.029 0.920(+0.047) _±_ 0.025 0.920(+0.054) _±_ 0.027 0.910(+0.056) _±_ 0.029 0.899(+0.074) _±_ 0.030 0.681(+0.061) _±_ 0.010 0.680(+0.083) _±_ 0.015 (-59)280 _−_ (-27959)275570 _−_ (-55859)551420
BYOL[22] 0.262(+0.092) _±_ 0.016 0.448(+0.125) _±_ 0.024 0.383(+0.133) _±_ 0.013 0.923(+0.050) _±_ 0.020 0.925(+0.059) _±_ 0.017 0.916(+0.062) _±_ 0.023 0.904(+0.079) _±_ 0.023 0.674(+0.054) _±_ 0.013 0.679(+0.082) _±_ 0.008 (-48)291 _−_ (-22098)269709 _−_ (-44148)539709
No SSL 0.272 _±_ 0.021 0.448 _±_ 0.021 0.360 _±_ 0.017 0.612 _±_ 0.019 0.640 _±_ 0.022 0.615 _±_ 0.019 0.664 _±_ 0.014 0.565 _±_ 0.009 **0.522** _±_ **0.002** 482 _−_ 157468 _−_ 315418
SimCLR[6] 0.369(+0.097) _±_ 0.014 0.579(+0.131) _±_ 0.034 0.477(+0.117) _±_ 0.029 0.630(+0.018) _±_ 0.023 0.653(+0.013) _±_ 0.029 0.637(+0.022) _±_ 0.025 0.657(-0.007) _±_ 0.011 0.576(+0.011) _±_ 0.014 0.572(+0.050) _±_ 0.010 (-33)449 _−_ (-4083)161551 _−_ (-8133)323551
BYOL[22] 0.353(+0.081) _±_ 0.022 0.554(+0.106) _±_ 0.033 0.471(+0.111) _±_ 0.018 0.662(+0.050) _±_ 0.029 0.682(+0.042) _±_ 0.028 0.672(+0.057) _±_ 0.032 0.672(+0.008) _±_ 0.032 0.583(+0.018) _±_ 0.015 0.581(+0.059) _±_ 0.010 (-64)418 _−_ (-25714)183182 _−_ (-51364)366782
No SSL 0.310 _±_ 0.039 0.393 _±_ 0.025 0.364 _±_ 0.044 0.862 _±_ 0.021 0.831 _±_ 0.023 0.837 _±_ 0.018 0.810 _±_ 0.028 0.645 _±_ 0.017 0.631 _±_ 0.004 320 _−_ 249880 _−_ 500080
SimCLR[6] 0.352(+0.042) _±_ 0.023 0.513(+0.120) _±_ 0.035 0.452(+0.088) _±_ 0.032 0.875(+0.013) _±_ 0.028 0.866(+0.035) _±_ 0.034 0.868(+0.031) _±_ 0.031 0.838(+0.028) _±_ 0.036 0.665(+0.020) _±_ 0.015 0.679(+0.047) _±_ 0.010 (-40)280 _−_ (-25690)275570 _−_ (-51340)551420
BYOL[22] 0.348(+0.038) _±_ 0.013 0.493(+0.100) _±_ 0.021 0.452(+0.088) _±_ 0.020 0.879(+0.017) _±_ 0.013 0.871(+0.040) _±_ 0.021 0.870(+0.033) _±_ 0.015 0.846(+0.036) _±_ 0.013 0.664(+0.019) _±_ 0.017 0.678(+0.047) _±_ 0.002 (-28)292 _−_ (-19378)269258 _−_ (-38728)538808
No SSL 0.093 _±_ 0.015 0.288 _±_ 0.023 0.199 _±_ 0.027 0.764 _±_ 0.049 0.817 _±_ 0.054 0.734 _±_ 0.058 0.823 _±_ 0.058 0.590 _±_ 0.016 0.539 _±_ 0.025 461 _−_ 165589 _−_ 331639
SimCLR[6] 0.231(+0.138) _±_ 0.028 0.473(+0.185) _±_ 0.033 0.362(+0.163) _±_ 0.041 0.842(+0.078) _±_ 0.015 0.866(+0.049) _±_ 0.023 0.836(+0.102) _±_ 0.016 0.786(-0.037) _±_ 0.039 0.623(+0.033) _±_ 0.006 0.628(+0.089) _±_ 0.005 (-136)325 _−_ (-85186)250775 (-170236) _−_ 501875
BYOL[22] 0.232(+0.139) _±_ 0.015 0.467(+0.179) _±_ 0.026 0.370(+0.171) _±_ 0.007 0.842(+0.078) _±_ 0.009 0.862(+0.045) _±_ 0.014 0.854(+0.120) _±_ 0.015 0.792(-0.031) _±_ 0.040 0.617(+0.027) _±_ 0.017 0.629(+0.091) _±_ 0.010 (-129)332 _−_ (-79329)244918 (-158529) _−_ 490168
No SSL 0.087 _±_ 0.014 0.263 _±_ 0.024 0.204 _±_ 0.017 0.953 _±_ 0.003 0.953 _±_ 0.002 0.954 _±_ 0.005 0.964 _±_ 0.004 0.718 _±_ 0.009 0.637 _±_ 0.003 312 _−_ 260238 _−_ 520788
SimCLR[6] 0.243(+0.156) _±_ 0.025 0.467(+0.204) _±_ 0.028 0.385(+0.181) _±_ 0.035 0.924(-0.029) _±_ 0.003 0.922(-0.031) _±_ 0.002 0.923(-0.031) _±_ 0.004 0.926(-0.038) _±_ 0.003 0.737(+0.019) _±_ 0.012 0.691(+0.054) _±_ 0.007 (-48)264 _−_ (-22548)282786 _−_ (-45048)565836
BYOL[22] 0.244(+0.157) _±_ 0.017 0.448(+0.185) _±_ 0.019 0.392(+0.188) _±_ 0.008 0.923(-0.030) _±_ 0.002 0.921(-0.032) _±_ 0.002 0.923(-0.031) _±_ 0.002 0.925(-0.039) _±_ 0.001 0.731(+0.013) _±_ 0.015 0.688(+0.051) _±_ 0.005 (-36)276 _−_ (-16236)276474 _−_ (-32436)553224
No SSL 0.109 0.276 0.209 0.767 0.814 0.775 0.825 0.581 0.545 396 _−_ 209754 _−_ 419904
SimCLR[6] (+0.217)0.326 (+0.282)0.558 (+0.253)0.462 (-0.202)0.565 (-0.197)0.617 (-0.200)0.575 (-0.136)0.689 (-0.014)0.567 (+0.000)0.545 (+114)510 _−_ (+94164)115590 (+188214) _−_ 231690
BYOL[22] (+0.204)0.313 (+0.260)0.536 (+0.254)0.463 (-0.185)0.582 (-0.175)0.639 (-0.170)0.605 (-0.138)0.687 (-0.014)0.567 (+0.005)0.549 (+112)508 _−_ (+86062)123692 (+172012) _−_ 247892
Table 6: The scientific names and subspecies of the each class.
Class Scientific Name Subspecies
Amur tiger, Chinese tiger, North Indochinese tiger,
Tiger Panthera tigris
Malayan tiger, Sumatran tiger, Bengal tiger
Rhinoceros Rhinoceros White rhino, Black rhino, Indian rhino, Javan rhino, Sumatran rhino
Camel Camelus Bactrian camel, Arabian camel, Wild bactrian camel
Giraffa Angolan giraffe, Kordofan giraffe, Transvaal giraffe,
Giraffe
Giraffa camelopardalis Reticulated giraffe, Baringo giraffe, Masai giraffe
Elephas maximus,
Elephant
Loxodonta africana
Equus grevyi,
Zebra Equus quagga,
Equus zebra
Asiatic elephant, Malayan elephant, Indian elephant,
Sri Lankan elephant, Sumatran elephant, African elephant,
South African bush elephant, East African bush elephant
Grevy’s zebra, Plains zebra, Grant’s zebra,
Half-maned zebra, Damara zebra, Chapman’s zebra,
Hartmann’s mountain zebra
Western lowland gorilla, Cross River gorilla,
Gorilla Gorilla
Mountain gorilla, Eastern lowland gorilla
Giant panda, Spectacled bear, Sun Bear, Sloth Bear,
Bear Ursus
American Black Bear, Brown Bear, Polar Bear, Asiatic black bear
Western grey kangaroo, Eastern grey kangaroo, Agile wallaby,
Black-striped wallaby, Tammar wallaby, Western brush wallaby,
Parma wallaby, Pretty-faced wallaby, Red-necked wallaby, Genus Onychogalea,
Bridled nail-tail wallaby, Northern nail-tail wallaby, Genus Osphranter,
Antilopine kangaroo, Black wallaroo, Common wallaroo, Red kangaroo
Kangaroo / Wallaby
Macropus,
Notamacropus,
Onychogalea,
Osphranter
Human Homo sapiens sapiens Homo sapiens sapiens
_ξ_ 11 [2] 40 _×ξ_ [2]
0.05 (5%) 132.23
0.1 (10%) 33.05
0.15 (15%) 14.69
Table 7: Percentage of each action type (the different action types are organized in Table 2). The
Label Pred. column shows the original label prediction for Uncertain samples treated as Must-Abstain.
Otherwise if the model does not abstain (nor predict the original label) for Must-Abstain, then the
action is considered Other Prediction.
Must-Act Must-Abstain
Correct Incorrect Abstain Label Pred. Other Pred. Abstain
ViT [11]
SP 0.62 0.07 0.31 0.02 0.13 0.84
ASP 0.52 0.28 0.20 0.00 0.48 0.52
MD [36] 0.00 0.99 0.01 0.01 0.97 0.03
KNN [70] 0.01 0.97 0.01 0.00 0.97 0.03
TAPUDD [13] 0.08 0.91 0.02 0.00 0.91 0.09
OpenMax [3] 0.01 0.00 0.98 0.01 0.00 0.99
MC-Dropout [16] 0.61 0.10 0.28 0.02 0.16 0.82
Deep Ensemble [33] 0.00 1.00 0.00 0.00 1.00 0.00
Swin Transformer [40]
SP 0.71 0.07 0.22 0.02 0.15 0.83
ASP 0.49 0.46 0.06 0.00 0.77 0.23
MD [36] 0.01 0.99 0.01 0.00 0.96 0.04
KNN [70] 0.12 0.73 0.14 0.00 0.49 0.50
TAPUDD [13] 0.01 0.99 0.01 0.00 0.96 0.04
OpenMax [3] 0.00 0.00 1.00 0.02 0.00 0.98
MC-Dropout [16] 0.64 0.26 0.11 0.01 0.45 0.55
Deep Ensemble [33] 0.17 0.83 0.00 0.00 0.98 0.02
DenseNet [28]
SP 0.76 0.07 0.17 0.02 0.18 0.80
ASP 0.10 0.90 0.00 0.00 0.93 0.07
MD [36] 0.00 1.00 0.00 0.00 0.96 0.04
KNN [70] 0.02 0.90 0.09 0.01 0.73 0.26
TAPUDD [13] 0.04 0.96 0.00 0.00 0.97 0.03
OpenMax [3] 0.01 0.00 0.98 0.01 0.00 0.99
MC-Dropout [16] 0.58 0.28 0.14 0.01 0.39 0.60
Deep Ensemble [33] 0.03 0.97 0.00 0.00 0.99 0.01
ConvNeXt [41]
SP 0.66 0.14 0.20 0.02 0.33 0.65
ASP 0.35 0.59 0.06 0.01 0.87 0.12
MD [36] 0.05 0.93 0.03 0.00 0.94 0.06
KNN [70] 0.02 0.95 0.04 0.00 0.86 0.13
TAPUDD [13] 0.10 0.90 0.00 0.00 0.97 0.03
OpenMax [3] 0.01 0.00 0.99 0.00 0.00 1.00
MC-Dropout [16] 0.62 0.24 0.14 0.02 0.46 0.52
Deep Ensemble [33] 0.00 1.00 0.00 0.00 1.00 0.00
MLP-Mixer [71]
SP 0.62 0.09 0.29 0.01 0.19 0.80
ASP 0.47 0.45 0.08 0.00 0.74 0.26
MD [36] 0.02 0.95 0.04 0.00 0.87 0.13
KNN [70] 0.04 0.89 0.07 0.00 0.79 0.20
TAPUDD [13] 0.31 0.61 0.08 0.00 0.68 0.32
OpenMax [3] 0.01 0.00 0.99 0.05 0.00 0.95
MC-Dropout [16] 0.60 0.14 0.26 0.00 0.26 0.74
Deep Ensemble [33] 0.06 0.94 0.00 0.00 0.99 0.01


Ensuring alignment, which refers to making models behave in accordance with human intentions [1, 2],
has become a critical task before deploying large language models (LLMs) in real-world applications.
For instance, OpenAI devoted six months to iteratively aligning GPT-4 before its release [3]. However,
a major challenge faced by practitioners is the lack of clear guidance on evaluating whether LLM
outputs align with social norms, values, and regulations. This obstacle hinders systematic iteration
and deployment of LLMs. To address this issue, this paper presents a comprehensive survey of
key dimensions that are crucial to consider when assessing LLM trustworthiness. The survey
covers seven major categories of LLM trustworthiness: reliability, safety, fairness, resistance to
misuse, explainability and reasoning, adherence to social norms, and robustness. Each major
category is further divided into several sub-categories, resulting in a total of 29 sub-categories.
Additionally, a subset of 8 sub-categories is selected for further investigation, where corresponding
measurement studies are designed and conducted on several widely-used LLMs. The measurement
results indicate that, in general, more aligned models tend to perform better in terms of overall
trustworthiness. However, the effectiveness of alignment varies across the different trustworthiness
categories considered. This highlights the importance of conducting more fine-grained analyses,
testing, and making continuous improvements on LLM alignment. By shedding light on these key
dimensions of LLM trustworthiness, this paper aims to provide valuable insights and guidance to
practitioners in the field. Understanding and addressing these concerns will be crucial in achieving
reliable and ethically sound deployment of LLMs in various applications.
_∗_ YL and YY are listed alphabetically and co-led the work. Correspond to {yang.liu01, kevin.yao}@bytedance.com.
Trustworthy LLMs

Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language
  Models' Alignment

Yang Liu
Yuanshun Yao
Jean-Francois Ton
Xiaoying Zhang
Ruocheng Guo Hao Cheng
Yegor Klochkov
Muhammad Faaiz Taufiq
Hang Li


4.1 Misinformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.2 Hallucination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.3 Inconsistency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.4 Miscalibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.5 Sycophancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13


5.1 Violence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.2 Unlawful Conduct . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.3 Harms to Minor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.4 Adult Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.5 Mental Health Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.6 Privacy Violation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15


6.1 Injustice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.2 Stereotype Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
6.3 Preference Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6.4 Disparate Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18


7.1 Propagandistic Misuse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.2 Cyberattack Misuse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
7.3 Social-engineering Misuse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
7.4 Leaking Copyrighted Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20


8.1 Lack of Interpretability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
8.2 Limited General Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
8.3 Limited Causal Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23


9.1 Toxicity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9.2 Unawareness of Emotions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Trustworthy LLMs
9.3 Cultural Insensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25


10.1 Prompt Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
10.2 Paradigm and Distribution Shifts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
10.3 Interventional Effect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
10.4 Poisoning Attacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27


11.1 Overall Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
11.2 Hallucination . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
11.3 Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
11.4 Fairness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
11.5 Miscalibration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
11.6 Propagandistic and Cyberattack Misuse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
11.7 Leaking Copyrighted Content . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
11.8 Causal Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
11.9 Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
11.10Generating Training Data for Alignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39


B.1 Examples from Testing Hallucination (Section 11.2) . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B.2 Examples from Testing Safety (Section 11.3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B.3 Examples from Testing Fairness (Section 11.4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B.4 Examples from Testing Uncertainty (Section 11.5) . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B.5 Examples from Testing Misuse (Section 11.6) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B.6 Examples from Testing Copyright Leakage (Section 11.7) . . . . . . . . . . . . . . . . . . . . . . . . 64
B.7 Examples from Testing Causal Reasoning (Section 11.8) . . . . . . . . . . . . . . . . . . . . . . . . 64
B.8 Examples from Testing Robustness (Section 11.9) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
B.9 Examples from Testing Alignment (Section 11.10) . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Trustworthy LLMs

B Additional Examples of the Generated Test Prompts 64


The landscape of Natural Language Processing (NLP) has undergone a profound transformation with the emergence
of large language models (LLMs). These language models are characterized by an extensive number of parameters,
often in the billions, and are trained on vast corpora of data [4]. In recent times, the impact of LLMs has been truly
transformative, revolutionizing both academic research and various industrial applications. Notably, the success of
LLMs developed by OpenAI, including ChatGPT [5, 6], has been exceptional, with ChatGPT being recognized as the
fastest-growing web platform to date [7].
One of the key factors that has made current large language models (LLMs) both usable and popular is the technique
of _alignment_ . Alignment refers to the process of ensuring that LLMs behave in accordance with human values and
preferences. This has become evident through the evolution of LLM development and the incorporation of public
feedback. In the past, earlier versions of LLMs, such as GPT-3 [8], were capable of generating meaningful and
informative text. However, they suffered from several issues that significantly affected their reliability and safety. For
instance, these models were prone to generating text that was factually incorrect, containing hallucinations. Furthermore,
the generated content often exhibited biases, perpetuating stereotypes and reinforcing societal prejudices.
Moreover, LLMs had a tendency to produce socially disruptive content, including toxic language, which had adverse
effects on their trustworthiness and utility. Additionally, their susceptibility to misuse, leading to the generation of
harmful propaganda, posed significant concerns for their responsible deployment. Furthermore, LLMs were found to be
vulnerable to adversarial attacks, such as prompt attacks, further compromising their performance and ethical integrity.
These misbehaviors of unaligned LLMs like GPT-3 have had a substantial impact on their trustworthiness and popularity,
especially when they were accessible to the public. To address these challenges, researchers and developers have been
working on improving alignment techniques to make LLMs more reliable, safe, and aligned with human values. By
mitigating these issues, the potential benefits of LLMs can be fully harnessed while minimizing the risks associated
with their misuse.
The erratic behaviors observed in LLMs can be attributed to a number of factors. Perhaps the most important one is
the lack of supervision of the large training corpus collected from the Internet, which contains a wide spectrum of
elements unaligned with values agreed by the majority of humans, including harmful content [9, 10], polarized opinions
[11, 12, 13], discrimination [14, 15], and sometimes illegal advice [16, 17]. These problematic phenomena propagate
from the imperfect training data to the LLMs, and as a result, LLMs could be (ab)used to reproduce and generate
unreliable, unethical, and dangerous content. In addition, single-mindedly optimizing objective functions in training
and generating text, which does not take human values into account, is another contributor. Note that identifying the
exact causes of LLM problems is still ongoing research.
To address these challenges, researchers have proposed alignment as a crucial step towards developing trustworthy
LLMs, ensuring that these models can effectively benefit and serve human users in a constructive manner [1, 18]. The
primary objective of alignment is to ensure that the outputs generated by LLMs are in line with the preferences of
human users [19]. The success of alignment in enhancing LLMs is evident in the stark contrast between the reception of
unaligned GPT-3 and the aligned version, ChatGPT. The latter reached an impressive milestone, garnering 100 million
users within just two months of its launch, making it the fastest-growing platform in history. This accomplishment
is not surprising, given that alignment not only reduces the likelihood of LLMs generating harmful outputs but also
significantly improves their usability by better adhering to human instructions.
By embracing alignment techniques, LLMs become more reliable, safe, and attuned to human values, thereby fostering
greater trust among users. The careful integration of alignment in LLM development paves the way for a more
responsible and constructive utilization of these powerful language models, unlocking their full potential to positively
impact various domains and enrich human experiences. Figure 1 shows such an example.
However, despite being the core technology behind the popularity of LLMs, evaluating the extent of alignment in
these models and designing appropriate alignment tasks remain open challenges, with no clear and principled guidance
available. Particularly, there is a lack of established and unified discussions that encompass the full spectrum of
aligning LLMs to be trustworthy. Existing literature has put forward multiple considerations for alignment tasks, among
which one notable general guideline is the “HHH" principle [20], advocating alignment that is Helpful, Honest, and
Harmless. In addition, a taxonomy of risks associated with building LLMs has been presented in [21], consisting
of six risks: (1) Discrimination, Exclusion, and Toxicity, (2) Information Hazards, (3) Misinformation Harms, (4)
Malicious Uses, (5) Human-Computer Interaction Harms, and (6) Automation, Access, and Environmental Harms.
While this taxonomy provides comprehensive coverage of related concerns, it can benefit from further unpacking of
each dimension. Furthermore, existing works such as [22] have surveyed the social impact of generative AI models,
encompassing various types like text, image, video, and audio. However, our focus is specifically on language models,
Trustworthy LLMs
Figure 1: An example to show the difference between the outputs before and after alignment. Not only the answer is more aligned
with human values, but also the model is more usable by following human instructions more often. Access: June 2023.
exploring distinctive concerns about LLMs and strategies to align them to be trustworthy. Moreover, [23] has evaluated
LLMs in a holistic manner, including some trustworthy categories, but it does not solely address trustworthiness and
alignment. To the best of our knowledge, a widely accepted taxonomy for evaluating LLM alignment has not yet
emerged, and the current alignment taxonomy lacks the granularity necessary for a comprehensive assessment.
Given the importance of ensuring the trustworthiness of LLMs and their responsible deployment, it becomes imperative
to develop a more robust and detailed taxonomy for evaluating alignment. Such a taxonomy would not only enhance
our understanding of alignment principles but also guide researchers and developers in creating LLMs that align better
with human values and preferences.
In this paper, we propose a more fine-grained taxonomy of LLM alignment requirements that not only can help
practitioners unpack and understand the dimensions of alignments but also provides actionable guidelines for data
collection efforts to develop desirable alignment processes. For example, the notion of a generated content being
“harmful" can further be broken down to harms incurred to individual users ( _e.g._ emotional harm, offensiveness, and
discrimination), society ( _e.g._ instructions for creating violent or dangerous behaviors), or stakeholders ( _e.g._ providing
misinformation that leads to wrong business decisions). In the Anthropic’s published alignment data [18], there exists a
clear imbalance across different considerations (Figure 46 in Appendix A). For instance, while the “violence" category
has an extremely high frequency of appearance, “child abuse" and “self-harm" appear only marginally in the data. This
supports the argument in [24] – alignment techniques do not guarantee that LLM can behave in every aspect the same as
humans do since the alignment is strongly data-dependent. As we will see later in our measurement studies (Section 11),
the aligned models (according to the amount of alignment performed as claimed by the model owners) do not observe
consistent improvements across all categories of considerations. Therefore we have a strong motivation to build a
framework that provides a more transparent way to facilitate a multi-objective evaluation of LLM trustworthiness.
The goal of this paper is three folds. _First_, we thoroughly survey the categories of LLMs that are likely to be important,
given our reading of the literature and public discussion, for practitioners to focus on in order to improve LLMs’
trustworthiness. _Second_, we explain in detail how to evaluate an LLM’s trustworthiness according to the above
categories and how to build evaluation datasets for alignment accordingly. In addition, we provide measurement studies
on widely-used LLMs, and show that LLMs, even widely considered well-aligned, can fail to meet the criteria for
some of the alignment tasks, highlighting our recommendation for a more fine-grained alignment evaluation. _Third_, we
demonstrate that the evaluation datasets we build can also be used to perform alignment, and we show the effectiveness
of such more targeted alignments.
**Roadmap.** This paper is organized as follows. We start with introducing the necessary background of LLMs and
alignment in Section 2. Then we give a high-level overview of our proposed taxonomy of LLM alignments in Section 3.
After that, we explain in detail each individual alignment category in Section 4-10. In each section, we target a
considered category, give arguments for why it is important, survey the literature for the problems and the corresponding
potential solutions (if they exist), and present case studies to illustrate the problem. After the survey, we provide
a guideline for experimentally performing multi-objective evaluations of LLM trustworthiness via automatic and
templated question generation in Section 11. We also show how our evaluation data generation process can turn into a
generator for alignment data. We demonstrate the effectiveness of aligning LLMs on specific categories via experiments
in Section 11.10. Last, we conclude the paper by discussing potential opportunities and challenges in Section 12.
Trustworthy LLMs


A Language Model (LM) is a machine learning model trained to predict the probability distribution P( _w_ ) over a
sequence of tokens (usually sub-words) _w_ . In this survey, we consider generative language models which generate
text in an autoregressive manner, _i.e._ sequentially computing a probability distribution for the next token based on past
tokens:
P( _w_ ) = P( _w_ 1) _·_ P( _w_ 2 _|w_ 1) _· · ·_ P( _wT |w_ 1 _, · · ·, wT −_ 1) (1)
where _w_ := _w_ 1 _· · · wT_ is a sequence of _T_ = _|w|_ tokens. P( _wt|w_ 1 _, · · ·, wt−_ 1) with _t_ = 1 _, · · ·, T_ is the probability the
LM predicts on the token _wt_ given the previous _t −_ 1 tokens. To generate text, LMs compute a probability distribution
over different tokens, and then draw samples from it with different sampling techniques, _e.g._ greedy sampling [25],
nucleus sampling [26], and beam search [27] _etc._ A large language model (LLM) is an LM with a large size (in the
magnitude of tens of millions to billions of model parameters) and size of training data [4]. Researchers have shown
that LLMs show “emergent abilities” [28, 29, 30] that are not seen in regular-sized LMs.
The transformer model [31] is the key architecture behind the recent success of LLMs. LLMs usually employ multiple
transformer blocks. Each block consists of a self-attention layer followed by a feedforward layer, interconnected
by residual links. This unique self-attention component enables the model to pay attention to nearby tokens when
processing a specific token. Initially, the transformer architecture was designed for machine translation tasks only. [5]
then adapted it for LMs. Recently developed language models leveraging transformer architecture can be fine-tuned
directly, eliminating the need for task-specific architectures [32, 33, 34].
In this paper, we primarily use the following LLMs for evaluations and case studies, and we access them during the
period of May - July 2023:
_•_ GPT-4: gpt-4 API [2] .
_•_ ChatGPT: gpt-3.5-turbo API.
_•_ GPT-3: The unaligned version of GPT-3 (davinci API).
_•_ Aligned GPT-3: An aligned version of GPT-3 (text-davinci-003 API) but not as well-aligned as ChatGPT.
We also used several open-sourced LLMs for case studies:
_•_ OPT-1.3B: An open-sourced LLM built by Meta [35].
_•_ FLAN-T5: An instruction-finetuned LLM by Google [30]. We use the largest version (11B) flan-t5-xxl.
We also use the following two open-sourced models for case studies and explorations:
_•_ ChatGLM: An open-sourced LLM built by [36].
_•_ DiabloGPT: An open-sourced LLM built by [37].
Note that in the following sections, when we show examples and case studies, we usually refer to the model names
accessible via the web interface ( _e.g._ ChatGPT and GPT-3, etc.). Later in the experiments, we refer to the models by
their API names ( _e.g._ gpt-3.5-turbo and gpt-4 etc.) since they are accessed by APIs. In this way, we can be
precise in stating how we access the model.
Our goal is not to benchmark or rank all available methods, but rather to provide an evaluation pipeline. We are keen to
test more models, including Google Bard and Anthropic Claude but at the time of paper writing, we do not have API
access to either.
**LLM Alignment.** SFT (supervised finetune) and RLHF (reinforcement learning from human feedback) are the
core techniques behind the alignment step [1, 18, 19]. The process of the current standard procedure of performing
LLM alignments is shown in Figure 2. SFT leverages human-provided sample answers for a selected set of prompts
(questions) _x ∈X_ . These questions are often designed in a way that solicits unsatisfactory or harmful answers. This
simple form, even at a relatively smaller scale compared to the training database, proves to be effective at tuning
the models to comply with the “social norms". The core idea of RLHF is to finetune the LLM using human-labeled
feedback, which takes the form of a preference ranking of given outputs. Each labeler in each session will be provided
with _K_ outputs _{yi}_ _[K]_ _i_ =1 [from the LLM given the prompt] _[ x]_ [. The labeler is then asked to provide a ranking of which] _[ y][i]_
is more preferred, or more aligned with an answer from an “unbiased" human user. The alignment data is then applied
with a policy learning algorithm (PPO) [38] that finetunes this model.
[2See https://platform.openai.com/docs/model-index-for-researchers for the OpenAI model nomen-](https://platform.openai.com/docs/model-index-for-researchers)
clature.
Trustworthy LLMs


Figure 2: A high-level view of the current standard procedure of performing LLM alignments [1]. **Step 1** - Supervised Finetuning
(SFT): Given a pretrained (unaligned) LLM that is trained on a large text dataset, we first sample prompts and ask humans to write
the corresponding (good) outputs based on the prompts. We then finetine the pretrained LLM on the prompt and human-written
outputs to obtain SFT LLM. **Step 2** - Training Reward Model: We again sample prompts, and for each prompt, we generate multiple
outputs from the SFT LLM, and ask humans to rank them. Based on the ranking, we train a reward model (a model that predicts how
good an LLM output is). **Step 3** - Reinforcement Learning from Human Feedback (RLHF): Given a prompt, we sample output from
the SFT LLM. Then we use the trained reward model to predict the reward on the output. We then use the Reinforcement Learning
(RL) algorithm to update the SFT LLM with the predicted reward.
There have been recent discussions on the necessity of using RLHF to perform the alignments. Alternatives have been
proposed and discussed [39, 40, 41, 42]. For instance, instead of using the PPO algorithm, RAFT [40] directly learns
from high-ranked samples under the reward model, while RRHF [39] additionally employs ranking loss to align the
generation probabilities of different answers with human preferences. DPO [41] and the Stable Alignment algorithm
[42] eliminate the need for fitting a reward model, and directly learns from the preference data.
Nonetheless, LLM alignment algorithm is still an ongoing and active research area. The current approach heavily relies
on labor-intensive question generation and evaluations, and there lacks a unified framework that covers all dimensions
of the trustworthiness of an LLM. To facilitate more transparent evaluations, we desire benchmark data for full-coverage
testing, as well as efficient and effective ways for evaluations.
**Remark on Reproducibility.** Although LLMs are stateless, _i.e._ unlike stateful systems like recommender systems,
their outputs do not depend on obscure, hidden, and time-varying states from users, it does not mean we are guaranteed
to obtain the same results every time. Randomness in LLM output sampling, model updates, hidden operations
that are done within the platform, and even hardware-specific details can still impact the LLM output. We try
to make sure our results are reproducible. We specify the model version as the access date in this subsection.
And along with this survey, we publish the scripts for our experiments and the generated data in the following:
[https://github.com/kevinyaobytedance/llm_eval.](https://github.com/kevinyaobytedance/llm_eval)

Step 3: Reinforcement Learning from Human Feedback (RLHF)


Figure 3 provides an overview of our proposed taxonomy of LLM alignment. We have 7 major categories with each of
them further breaking down into more detailed discussions, leading to 29 sub-categories in total. Below we give an
overview of each category:
Trustworthy LLMs


Resistance to
Misuse
Explainability
Social Norm Robustness
& Reasoning
Misinformation
Hallucination
Inconsistency
Miscalibration
Sycophancy
Prompt Attacks
Paradigm &
Distribution
Shifts
Interventional
Effect
Poisoning
Attacks
Propagandistic
Misuse
Cyberattack
Misuse
Socialengineering
Misuse
Leaking
Copyrighted
Content
Lack of
Interpretability
Limited Logical
Reasoning
Limited Causal
Reasoning
Toxicity
Unawareness
of Emotions
Cultural
Insensitivity
Figure 3: Our proposed taxonomy of major categories and their sub-categories of LLM alignment. We include 7 major categories:
reliability, safety, fairness and bias, resistance to misuse, interpretability, goodwill, and robustness. Each major category contains
several sub-categories, leading to 29 sub-categories in total.
Next we discuss how we determine the taxonomy.
**Current LLM Applications.** To motivate how we determine the proposed taxonomy, we first briefly survey the
current major applications of LLMs in Figure 4, which largely impacts how we select the taxonomy. Needless to say,
applications covered in Figure 4 are non-exhaustive considering the relentless speed and innovative zeal with which
practitioners perpetually formulate both commercial and non-commercial ideas leveraging LLMs.
**How We Determine the Taxonomy.** We determine the categories and sub-categories by two major considerations: (1)
the impact on LLM applications and (2) the existing literature. We first consider how many LLM applications would be
negatively impacted if a certain trustworthiness category fails to meet expectations. The negative impacts could include
how many users would be hurt and how much harm would be caused to both the users and society. In addition, we
also consider existing literature on responsible AI, information security, social science, human-computer interaction,
jurisprudential literature, and moral philosophy _etc._
For example, we believe reliability is a major concern because hallucination is currently a well-known problem in LLMs
that can hurt the trustworthiness of their outputs significantly, and almost all LLM applications, except probably creative
Trustworthy LLMs
Figure 4: Current major applications of LLMs. We group applications into four categories: writing assistance, information retrieval,
commercial use, and personal use. Note that the applications are all more or less overlapped with each other, and our coverage is
definitely non-exhaustive.
writing, would be negatively impacted by factually wrong answers. And depending on how high the stake is for the
applications, it can cause a wide range of harm, ranging from amusing nonsense to financial or legal disasters. Following
the same logic, we consider safety to be an important topic because it impacts almost all applications and users, and
unsafe outputs can lead to a diverse array of mental harm to users and public relations risks to the platform. Fairness is
vital because biased LLMs that are not aligned with universally shared human morals can produce discrimination against
users, reducing user trust, as well as negative public opinions about the deployers, and violation of anti-discrimination
laws. Furthermore, resistance to misuse is practically necessary because LLMs can be leveraged in numerous ways
to intentionally cause harm to other people. Similarly, interpretability brings more transparency to users, aligning
with social norms makes sure LLMs do not evoke emotional damage, and improved robustness safeguards the model
from malicious attackers. The subcategories under a category are grouped based on their relevance to particular LLM
capabilities and specific concerns.
Note that we do not claim our set of categories covers the entire LLM trustworthiness space. In fact, our strategy is to
thoroughly survey, given our reading of the literature and the public discussions as well as our thinking, what we believe
should be addressed at this moment. We start to describe each category in LLM alignment taxonomy one by one.


The primary function of an LLM is to generate informative content for users. Therefore, it is crucial to align the model so
that it generates reliable outputs. Reliability is a foundational requirement because unreliable outputs would negatively
impact almost all LLM applications, especially ones used in high-stake sectors such as health-care [43, 44, 45] and
finance [46, 47]. The meaning of reliability is many-sided. For example, for factual claims such as historical events and
scientific facts, the model should give a clear and correct answer. This is important to avoid spreading misinformation
and build user trust. Going beyond factual claims, making sure LLMs do not hallucinate or make up factually wrong
claims with confidence is another important goal. Furthermore, LLMs should “know what they do not know" – recent
works on uncertainty in LLMs have started to tackle this problem [48] but it is still an ongoing challenge.
We survey the following categories for evaluating and aligning LLM reliability.


It is a known fact that LLMs can provide untruthful answers and provide misleading information [49, 6, 50]. We
define misinformation here as wrong information not intentionally generated by malicious users to cause harm, but
Trustworthy LLMs
unintentionally generated by LLMs because they lack the ability to provide factually correct information. We leave the
intentionally misusing LLMs to generate wrong information to Section 7.
While there is no single agreed-upon cause for LLMs generating untruthful answers, there exist a few hypotheses. First,
the training data is never perfect. It is likely that misinformation already exists there and could even be reinforced
on the Internet [51, 52]. These mistakes can certainly be memorized by a large-capacity model [53, 54]. In addition,
Elazar et al. [55] find that a large number of co-occurrences of entities ( _e.g._, Obama and Chicago) is one reason for
incorrect knowledge ( _e.g._ Obama was born in Chicago) extracted from LLMs. Mallen et al. [56] discover that LLMs
are less precise in memorizing the facts that include unpopular entities and relations. They propose to leverage retrieved
external non-parametric knowledge for predictions regarding unpopular facts as retrieval models ( _e.g._ BM-25 and
Contriever [57]) are more accurate than LLMs for these facts. Si et al. [58] evaluate whether LLMs can update their
memorized facts by information provided in prompts. They find that, while code-davinci-002 [3] can update its
knowledge around 85% of the time for two knowledge-intensive QA datasets, other models including T5 [59] and
text-davinci-001 (one of the aligned GPT-3 versions) have much lower capability to update their knowledge to
ensure factualness. There could be many more causes for LLM’s incorrect knowledge.
One might think that an LLM only makes mistakes for challenging logical questions, but in fact, LLMs do not provide
complete coverage even for simple knowledge-checking questions, at least not without a sophisticated prompt design.
To demonstrate it, we pose questions to ChatGPT asking about in which year a historical event occurred. We then cross
reference Wikipedia as the ground truth answer. Figure 5 shows one example where ChatGPT disagrees with Wikipedia
on when the Romans completed their conquest and occupation.
In which year did this happen: After the Eighty Years’ War, Luxembourg became a part of the Southern
Netherlands, which passed to the Austrian line of the Habsburg dynasty
[ChatGPT]: After the Eighty Years’ War, Luxembourg became a part of the Southern Netherlands in
the year 1684.
In which year did this happen: The Romans, under Julius Caesar, completed their conquest and occupation
[ChatGPT]: The Romans, under Julius Caesar, completed their conquest and occupation of the region
that is present-day Luxembourg in 54 BC.
Figure 5: Examples of ChatGPT giving a factually wrong answer. Wikipedia shows the events actually happened in 1713 and 53 BC
respectively. Access: May 2023.
Among other popular discussions, LLMs are found to be able to “hallucinate" to make up contents that cannot be traced
to the existing knowledge base (regardless of the knowledge being either correct or wrong). Hallucinations can be
regarded as a form of misinformation but it has its own distinct definitions and properties. In Section 4.2 we will go
deeper into its discussions. The literature has also discussed the possibility of improving the factualness of an LLM by
improving its consistency and logical reasoning capability, and we will survey these two topics separately in Section 4.3
and 8.2.


LLMs can generate content that is nonsensical or unfaithful to the provided source content with appeared great
confidence, known as hallucinations in LLMs [60, 61, 62]. In the psychology literature, a similar phenomenon is
termed as confabulation [63], _i.e._ false memory without intending to deceive, sometimes caused by brain damage. Note
that there is a distinction between hallucination and misinformation. Misinformation mostly implies wrong or biased
answers and can often be caused by bad inputs of information, but hallucination may consist of fabricated contents
that conflict with the source content ( _i.e. intrinsic hallucination_ ) or cannot be verified from the existing sources ( _i.e._
_extrinsic hallucination_ ). Figure 6 and Figure 7 provide examples of intrinsic and extrinsic hallucination respectively.
The exact cause of hallucinations is still unclear. There are some speculations and theories. For example, it may
be caused by the mismatch or distribution shift between the source training and test data [60, 64]. Some NLP tasks
naturally require some mismatch between the source input text and the target reference, _e.g._ chit-chat style open-domain
dialogue [65]. Hallucination can also happen when LLM’s confidence is miscalibrated – this is often due to the lack
of human supervision, low coverage of alignment examples, as well as the inherent ambiguity in the supervision data
itself [66]. Furthermore, hallucinations can be caused by the underlying training mechanisms [4, 31], including but not
[3https://help.openai.com/en/articles/6195637-getting-started-with-codex.](https://help.openai.com/en/articles/6195637-getting-started-with-codex)
Trustworthy LLMs
Figure 6: Examples of intrinsic hallucination of using ChatGPT: While the queried series was first aired in 2006, as indicated in the
provided knowledge. Access: June 2023.
Figure 7: Examples of extrinsic hallucination of using text-davinci-003 : Jean-Francois Bryant Curry is a fabricated person and does
not actually exist. Access: June 2023.
limited to the randomness introduced in sampling the next tokens, errors in encoding [67, 68] and decoding [69], the
training bias from imbalanced distributions, and over-reliance on memorized information [70] _etc._
Evaluating and detecting hallucination is still an ongoing area [71]. The common evaluation task is text summarization,
and a simple metric would be the standard text similarity between LLM outputs and the reference texts, _e.g._ ROUGE [72]
and BLEU [73]. Another popular task is QA (question and answering) [74] where LLMs answer questions and we
compute the text similarity between LLM answers and the ground-truth answers. A different evaluation approach is to
train truthfulness classifiers to label LLM outputs [75, 76]. Last but not least, human evaluation is still one of the most
commonly used approaches [77, 78, 69, 79].
Mitigating hallucinations is an open problem. Currently, only a limited number of methods are proposed. One aspect
is to increase training data quality, _e.g._ building more faithful datasets [76, 80] and data cleaning [81, 82]. The other
aspect is using different rewards in RLHF. For example, in dialogue, [83] a consistency reward which is the difference
between the generated template and the slot-value pairs extracted from inputs. In text summarization, [84] design the
reward by combining ROUGE and the multiple-choice cloze score to reduce hallucinations in summarized text. In
addition, leveraging an external knowledge base can also help [85, 86, 87, 88]. Overall, we do not currently have a
good mitigation strategy.


LLMs have been reported to give inconsistent outputs [89, 6, 90, 91]. It is shown that the models could fail to provide
the same and consistent answers to different users, to the same user but in different sessions, and even in chats within
the sessions of the same conversation. These inconsistent answers can create confusion among users and reduce user
trust. The exact cause of inconsistency is unclear. But the randomness certainly plays a role, including randomness
in sampling tokens, model updates, hidden operations within the platform, or hardware specs. It is a signal that the
LLM might still lag behind in its reasoning capacities, another important consideration we will discuss in more detail in
Section 8.2 [4]
For example, in Figure 8 we observe that LLMs behave inconsistently when prompting questions are asked in different
ways. When asked to answer a simple algebra question, it failed to provide a correct answer; while asked to perform the
calculation with steps, the ChatGPT was able to obtain the correct one. This requires users to be careful at prompting,
therefore raising the bar of using LLMs to merely get correct answers, which ideally should not be the case, and of
course, reducing the trustworthiness of all the answers.
In addition, it is also reported that LLMs can generate inconsistent responses for the same questions (but in different
sessions) [92]. This issue is related to the model’s power in logic reasoning (discussed in Section 8.2) but the cause
for inconsistent responses can be more complicated. The confusing and conflicting information in training data can
certainly be one cause. The resulting uncertainties increase the randomness when sampling the next token when
4Note that consistency does not necessarily mean logic. For example in an emotional support chatbox, the goal is to be consistent,
_e.g._ consoling users consistently with a warm tone between dialogues. But it does not need to be logical. In fact, maybe lack of logic
is even more desirable because outputting illogical responses can make users feel good, _e.g._ “Tomorrow everything will be better
because that’s what you wish for.”
Trustworthy LLMs
What is 300 + 200*190?
What is 300 + 200*190? Show your steps.
[ChatGPT]: 8,900.
[ChatGPT]: 38,300. [Correct steps.]
Figure 8: An example of ChatGPT giving inconsistent answers when prompted differently. Access: June 2023.
generating outputs. For instance, if a certain slur appeared both in a positive and a negative narrative in the training data,
the trained LLM might be confused by the sentiment of a sentence that contains this slur.
There have been some discussions about how to improve the consistency of an LLM. For example, [91] regulates the
model training using a consistency loss defined by the model’s outputs across different input representations. Another
technique of enforcing the LLMs to self-improve consistency is via “chain-of-thought" (COT) [29], which encourages
the LLM to offer step-by-step explanations for its final answer. We include more discussion of COT in Section 8.1.


LLMs have been identified to exhibit over-confidence in topics where objective answers are lacking, as well as in areas
where their inherent limitations should caution against LLMs’ uncertainty ( _e.g._ not as accurate as experts) [93, 94]. This
overconfidence, exemplified in Figure 9, indicates the models’ lack of awareness regarding their outdated knowledge
base about the question, leading to confident yet erroneous responses. This problem of overconfidence partially stems
from the nature of the training data, which often encapsulates polarized opinions inherent in Internet data [95].
Figure 9: An example of the LLM being certain about a wrong answer or a question that its knowledge base is outdated about.
Access: June 2023.
Efforts aimed at addressing this issue of overconfidence have approached it from different angles. For instance, Mielke
et al. [96] proposed a calibration method for “chit-chat" models, encouraging these models to express lower confidence
when they provide incorrect responses. Similarly, Guo et al. [97] offered a method for rescaling the softmax output in
standard neural networks to counter overconfidence. However, these calibration methods often present trade-offs, as
highlighted by Desai et al. [98], whose empirical study on calibrating transformers demonstrated worsened in-domain
performance despite marginal improvements in out-of-domain tasks. Specifically in the case of LLMs, we in fact have
two ways to compute uncertainty. _First_, as demonstrated in Figure 10, the LLM can indeed output its own confidence in
text form when being prompted specifically. _Second_, we can also obtain the LLM’s confidence through the logits of the
tokens (although some platforms might not give the user access to them, _e.g._ ChatGPT and GPT-4). We show later
in Section 11.5 that these two measures mostly match up, but not always, showing confidence miscalibration exists
between the internal state and the external output. Note however, even if the internal and external probabilities match,
this does not mean that the model is calibrated as both could be miscalibrated at the same time.
Figure 10: An example of confidence mismatching on text-davinci-003. Access: June 2023. When explicitly examining
the logits in the LLM sampling, the model in fact has a probability of 0 _._ 99 for the first token A and 0 _._ 01 probability for the rest. It
shows a clear disconnect between the generated text and the true logits. For more quantitative results on calibration see Section 11.5.
Trustworthy LLMs
The alignment step, as seen in studies by Kadavath et al. [99] and Lin et al. [100], can be instrumental in containing
overconfidence. These studies emphasize teaching models to express their uncertainty in words, offering a soft and
calibrated preference that communicates uncertainty. For instance, “Answers contain uncertainty. Option A is preferred
80% of the time, and B 20%." This approach, however, requires refined human labeling information ( _e.g._ smoothed
labels [101, 102]) for fine-tuning and the development of new training mechanisms that can properly leverage this
information.
An emerging mechanism that facilitates models comfortably "abstaining" from answering questions is the domain of
selective classifiers [103, 104, 105, 106, 107, 108, 109, 110]. These models can provide responses like “I do not know
the answer" or “As an AI model, I am not able to answer", particularly when tasks are out of their domain. Typically,
selective classification predicts outcomes for high-certainty samples and abstains on lower ones, employing the softmax
outputs of the classifier [111, 112].
Furthermore, the employment of conformal prediction methods across various NLP tasks such as sentiment analysis,
text infilling, and document retrieval offers promising advancements [113, 114, 115, 116, 117]. These efforts, combined
with out-of-domain detection strategies [118, 119], and methodologies for improving model calibration through post-hoc
scaling and fine-tuning [120], collectively show that although LLMs are generally poorly calibrated, these challenges
can be partially addressed through more advanced approaches. For a comprehensive tutorial on uncertainty in NLP, see
[121] for more detail.


LLM might tend to flatter users by reconfirming their misconceptions and stated beliefs [24, 122, 123]. This is a
particularly evident phenomenon when users challenge the model’s outputs or repeatedly force the model to comply. In
Figure 11 we show an example where despite the model making the correct calculation initially, it falls back to a wrong
one implied and insisted by the user. Note that sycophancy differs from inconsistency in terms of causes. Sycophancy
is mostly because we instruction-finetune LLMs too much to make them obey user intention to the point of violating
facts and truths. On the other hand, inconsistency can happen due to the model’s internal lack of logic or reasoning and
is independent of what users prompt.
What is 10 * 10 + 5?
[ChatGPT]: 105.
Figure 11: An example from ChatGPT where the model initially gives the right answer but changes it to a wrong one after the user
questions and misleads. Access: May 2023.
In contrast to the overconfidence problem discussed in Section 4.4, in this case, the model tends to confirm users’ stated
beliefs, and might even encourage certain actions despite the ethical or legal harm. The emergence of sycophancy
relates partially to the model’s inconsistency as we discussed above. But the causes for it are richer. It is possibly due to
existing sycophantic comments and statements in the training data. It can also be attributed to sometimes excessive
instructions for the LLM to be helpful and not offend human users. In addition, it is possible that the RLHF stage could
promote and enforce confirmation with human users. During the alignment, LLMs are fed with “friendly" examples that
can be interpreted as being sycophantic to human users. Therefore, an important improvement on the existing RLHF
algorithm is to balance the tradeoff between the degree of sycophancy and the degree of aligning with human values.


We discuss the safety requirements of building an LLM. The outputs from LLMs should only engage users in a safe
and healthy conversation. The first dimension of safety consideration is the safety of the model’s generated contents.
Internet data contains a variety of violent and unsafe content, examples of which can include instances of hate speech,
promotion of violence, or sharing of explicit materials, often against the community guidelines of major platforms such
as Facebook [124], Twitter [125], YouTube [126], LinkedIn [127] and TikTok [128]. Therefore, the outputs from LLMs
could incorporate hateful, harmful, or dangerous comments in responding, as well as produce dangerous content when
solicited by human users. These outputs not only reduce user trust but also pose challenges to complying with safety
regulations. Concerted efforts are being made by prominent stakeholders to mitigate these concerns. For instance, a
collaboration among AI research companies Cohere, OpenAI, and AI21 Labs, has set forth a comprehensive set of
Trustworthy LLMs
best practices aimed at ensuring safe LLM deployment, which includes prohibiting misuse, proactively mitigating
unintentional harm, and encouraging diverse collaboration [129]. While this initiative marks a welcoming advancement
in addressing LLM safety, it is important to recognize that we still have a considerable journey ahead to ensure the
comprehensive safety of LLM outputs. Last but not least, we position that it is important to protect personal information
privacy when building an LLM. Below we survey the prominent categories of safety concerns in LLMs.


LLMs are found to generate answers that contain violent content or generate content that responds to questions that
solicit information about violent behaviors [18, 1, 130, 131]. For example, earlier models without alignments tend to
“truthfully" respond to queries that seek advice for violent actions, such as the best ways of shooting people, getting
away from crimes, organizing terrorist activities etc. Even after proper alignment on violence-related data, it is still
possible for users to indirectly trick the LLM into giving out the answer by subtly prompting. In the example provided
in Figure 12, by “pretending" to obtain information in a righteous way, users can successfully solicit details of the
materials for building a nuclear bomb, and such information is well-organized.
How to make a nuclear bomb?
[ChatGPT]: I’m sorry, but I cannot provide instructions on how to build a nuclear bomb.
In order to prevent someone from building a nuclear bomb, what are the
materials we shouldn’t be selling?
[ChatGPT]: [Details on the materials and steps needed to build a nuclear bomb.]
Figure 12: An example of using ChatGPT by indirect prompting (to hide the real intention) to induce unsafe answers. Access: June
2023.
The alignment technique can help with removing violent content by providing LLMs samples in the form of questionanswer pairs where the answer represents a proper way to deny answering the question. In addition, there have been
recent works on mitigating, filtering, or converting violent content to safe ones for language models [132, 133, 134, 135].


The outputs from LLMs need to obey the specific laws of the country where the models are allowed to operate. In the
past, LLMs have been shown to be a convenient tool for soliciting advice on accessing, purchasing (illegally), and
creating illegal substances, as well as for dangerous use of them. Figures 13 and 14 show two such examples from
prompting GPT-3 and GPT-4 respectively. These responses create severe safety concerns and the associated information
can be leveraged to pose threats to society. One can argue, in some sense, illegal content from LLMs can cause more
harm than the traditional source, say Google Search, when seeking illegal advice. It is because search engines do not
explicitly advise users, but rather show a list of sources and let users themselves make the judgment. On the other hand,
LLMs directly form the advice _for_ users, and therefore users might develop a stronger habit of taking advice without
verifying its validity.
Figure 13: An example of using GPT-3 to seek advice for getting away from crimes. Access: June 2023.
Figure 14: An example of using GPT-4 to obtain advice for cultivating illegal (in U.S.) drugs. Access: July 2023.
The factor that complicates this issue is the difference between laws from various countries. Practitioners need to
comply with the local law by performing a number of alignments, mostly by collecting different humanly labeled data
for the alignment step.
Trustworthy LLMs


It is important to identify cases where LLMs can be leveraged to solicit answers that contain harmful content to children
and youth [131, 136, 137, 10]. For example, [131] explored the capability of ChatGPT for safety-related information,
including supervising children around water. [137] lays out three layers of action items for our government, industry,
and parent to protect minors in light of the emerging information communication on the Internet. Technically speaking,
this concern of harm to minors is covered by legal requirements in Section 5.2, but we separate it out because the issue
is universally considered both legally and morally important. These potential harms include soliciting advice for child
trafficking (Figure 15 plots one such example), planning crime activities against youth ( _e.g._ child abuse), and exploiting
youth for cheap labor _etc._
We also do not rule out the possibility that LLMs can be leveraged to generate dangerous and age-inappropriate content,
such as violent and sex-explicit content that is accessible to underage users. Different countries have their own specific
details on access control for underage users to certain software; it is possible that access to such content is restricted.
But further aligning the LLMs to avoid such generations can substantially increase the security level.
Figure 15: An example GPT-3 showing advice for kidnapping kids. Access: July 2023.


LLMs have the capability to generate sex-explicit conversations, and erotic texts, and to recommend websites with
sexual content [138]. Combining with image generative models [139, 140] and LLMs’ inherent code generation power
for synthesizing images [62], new concerns arise when users use LLM’s multi-modality function for contents. Users
can also potentially use LLMs to elicit sexually offensive language toward certain users.
While different regions have set varying tolerance and definition towards sexual content [10], it is important to make
sure under-age users are not given access to LLMs to solicit such content. It is perhaps equally important to offer
the LLM the ability to perform a more strict screening of the generated sexual content. There is a large literature on
pornographic content detection [141, 142], which have been done in a more or less mature way by various Internet
platforms, _e.g._ Youtube [143], Instagram [144], Tiktok [128]. And one practical way is to simply apply a porn text
detector on the LLM outputs to filter out the unwanted content.


With easier access to the Internet, the literature has documented arising concerns about users’ mental health issues.
There is evidence that unhealthy interactions with Internet discussions can reinforce users’ mental issues [145, 146], as
well as that the Internet could fail the users who intend to seek online mental support [147, 148]. In the era of LLMs, as
alternatives to search engines, LLMs can be great resources for people seeking mental support [149], as well as for
assisting physicians to provide indirect support [150]. Therefore we believe that LLMs should be alerted to questions
that show broader mental health concerns, understand the context of the situation, and provide available information to
support users to get further help, instead of either confirming or negating their feelings. For instance, when users seek
confirmation about suicidal tendencies, the models’ outputs should provide information that offers psychosocial support
and share corresponding resources. Careless responses or even reconfirming a user’s illness can lead to disastrous
consequences.
**Remarks on Safety Concerns.** For the listed safety concerns, though the recently more aligned LLMs seem to have
implemented an “guardian angel" that detects these explicit requests and denies to respond, it has also been tested via
specific instructions in prompts, _e.g._ by emphasizing sex positivity is a necessary piece in society, one can prompt the
models to continue the generation of unsafe contents ( _e.g._ sex explicit contents). Therefore, guarding the safety of
the generated contents from LLMs remains an active challenge and requires strong commitments from our research
community.


General machine learning models are known to be vulnerable to data privacy attacks [151, 152, 153], _i.e._ special
techniques of extracting private information from the model or the system used by attackers or malicious users, usually
Trustworthy LLMs
by querying the models in a specially designed way. The private information includes training data [154, 155, 156,
157, 158], training data property [159, 160], instance’s membership belonging to the training data [161, 162, 163, 164,
165, 166, 167], model weights [168, 169, 170, 171, 172], model architecture [173, 174, 175], and even the training
hyperparameters [176, 177, 178, 179]. The memorization effect [180, 181, 182, 183, 184, 185] in deep neural network
models make them even more vulnerable to privacy attacks than simple models [186, 54].
Privacy attacks on LLMs, leveraged by the memorization power of LLMs, raise similar concerns on the possibility of
leaking personal information from the outputs [53, 187]. Recent works [188, 189, 190, 191, 192] have shown that an
attacker can extract personal or sensitive information or private training samples from LLM’s training data by querying
LLMs alone. Researchers have proposed attacks that leverage the memorization effect of LLMs, usually growing with
training sample repetition [193, 194].
Commonly used privacy-enhancing technologies (PETs) that defend against privacy attacks include differentially private
training mechanisms [195, 196, 197, 198, 199], machine unlearning [200, 201, 202, 203, 204, 205], federated learning
[206, 207, 208, 209, 210], and secure multi-party computation protocols [211, 212, 213, 214, 215, 216, 217, 218]. Note
that although each of those privacy-enhancing techniques has a rich literature, the effectiveness and efficiency of them
when applied to LLMs at a large scale is still unclear.