|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- grammarly/pseudonymization-data |
|
|
language: |
|
|
- en |
|
|
metrics: |
|
|
- f1 |
|
|
- bleu |
|
|
pipeline_tag: text2text-generation |
|
|
--- |
|
|
|
|
|
# Model Card for Model ID |
|
|
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
This repository contains files for two Seq2Seq transformers-based models used in our paper: https://arxiv.org/abs/2306.05561. |
|
|
|
|
|
## Model Details |
|
|
|
|
|
### Model Description |
|
|
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
|
|
- **Developed by:** Oleksandr Yermilov, Vipul Raheja, Artem Chernodub |
|
|
- **Model type:** Seq2Seq |
|
|
- **Language (NLP):** English |
|
|
- **License:** Apache license 2.0 |
|
|
- **Finetuned from model:** BART |
|
|
|
|
|
### Model Sources |
|
|
|
|
|
- **Paper:** https://arxiv.org/abs/2306.05561 |
|
|
|
|
|
## Uses |
|
|
|
|
|
These models can be used for anonymizing datasets in English language. |
|
|
|
|
|
## Bias, Risks, and Limitations |
|
|
|
|
|
Please check the Limitations section in our paper. |
|
|
|
|
|
|
|
|
## Training Details |
|
|
|
|
|
### Training Data |
|
|
|
|
|
https://huggingface.co/datasets/grammarly/pseudonymization-data/tree/main/seq2seq |
|
|
|
|
|
### Training Procedure |
|
|
|
|
|
1. Gather text data from Wikipedia. |
|
|
2. Preprocess it using NER-based pseudonymization. |
|
|
3. Fine-tune BART model on translation task for translating text from "original" to "pseudonymized". |
|
|
|
|
|
|
|
|
#### Training Hyperparameters |
|
|
|
|
|
We train the models for 3 epochs using `AdamW` optimization with the learning rate α =2*10<sup>5</sup>, and the batch size is 8. |
|
|
|
|
|
## Evaluation |
|
|
|
|
|
### Factors & Metrics |
|
|
|
|
|
#### Factors |
|
|
|
|
|
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. --> |
|
|
|
|
|
There is no source truth of named entities for the data, on which this model was trained. We check whether the word is a named entity, using one of the NER systems (spaCy or FLAIR). |
|
|
|
|
|
#### Metrics |
|
|
|
|
|
|
|
|
We measure the amount of text, changed by our model. Specifically, we check for the following categories of translated text word by word: |
|
|
1. True positive (TP) - Named entity, which was changed to another named entity. |
|
|
2. True negative (TN) - Not a named entity, which was not changed. |
|
|
3. False positive (FP) - Not a named entity, which was changed to another word. |
|
|
4. False negative (FN) - Named entity, which was not changed to another named entity. |
|
|
We calculate F<sub>1</sub> score based on the abovementioned values. |
|
|
|
|
|
|
|
|
## Citation |
|
|
|
|
|
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. --> |
|
|
|
|
|
**BibTeX:** |
|
|
|
|
|
``` |
|
|
@misc{yermilov2023privacy, |
|
|
title={Privacy- and Utility-Preserving NLP with Anonymized Data: A case study of Pseudonymization}, |
|
|
author={Oleksandr Yermilov and Vipul Raheja and Artem Chernodub}, |
|
|
year={2023}, |
|
|
eprint={2306.05561}, |
|
|
archivePrefix={arXiv}, |
|
|
primaryClass={cs.CL} |
|
|
} |
|
|
``` |
|
|
|
|
|
## Model Card Contact |
|
|
|
|
|
Oleksandr Yermilov ([email protected]). |
