README.md

---
library_name: transformers
license: mit
datasets:
- SemEvalWorkshop/sem_eval_2010_task_8
language:
- en
base_model:
- google/flan-t5-base
---

# Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->



## Model Details

### Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Sefika Efeoglu

- **Model type:** text-to-text
- **Language(s) (NLP):** [More Information Needed]
- **License:** [More Information Needed]
- **Finetuned from model [optional]:** https://huggingface.co/google/flan-t5-base


## Uses

```python
import json
import torch
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM
from datetime import datetime
from transformers import T5Tokenizer, T5ForConditionalGeneration

tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-base")
model_id = "Sefika/semeval_prompt_tuning_5"
model = T5ForConditionalGeneration.from_pretrained(model_id,
                                                    device_map="auto",
                                                    load_in_8bit=False,
                                                    torch_dtype=torch.float16)
prompt = """Example Sentence:The purpose of the <e1>audit</e1> was to report on the <e2>financial statements</e2>.\n"""+\
         """Sentence: Query Sentence:The most common <e1>audits</e1> were about <e2>waste</e2> and recycling.\n"""+\
         """What is the relation type between e1: audits. and e2 : waste.  according to given relation types below in the sentence?\n"""+\
         """Relation types: Relation types: Cause-Effect(e2,e1), Content-Container(e1,e2), Member-Collection(e1,e2), Instrument-Agency(e1,e2), Product-Producer(e2,e1), Member-Collection(e2,e1), Message-Topic(e1,e2), Entity-Origin(e2,e1), Message-Topic(e2,e1), Instrument-Agency(e2,e1), Content-Container(e2,e1), Product-Producer(e1,e2), Entity-Origin(e1,e2), Component-Whole(e1,e2), Entity-Destination(e1,e2), Other, Cause-Effect(e1,e2), Component-Whole(e2,e1), Entity-Destination(e2,e1). \n"""
inputs = self.tokenizer(prompt, add_special_tokens=True, max_length=526,return_tensors="pt").input_ids.to("cuda")

outputs = self.model.generate(inputs, max_new_tokens=length,  pad_token_id=self.tokenizer.eos_token_id)

response = self.tokenizer.batch_decode(outputs, skip_special_tokens=True)
print(response[0])
#"Cause-Effect(e1,e2)"

```


## Training Details

### Training Data

semeval-2010-task8
[More Information Needed]

### Training Procedure

5 fold cross validation with sentence and relation types. Input is sentence and the output is relation types

#### Training Hyperparameters

Epoch:5, BS:16 and others are default.

#### Hardware

Colab Pro+ A100.



## Citation [optional]

Efeoglu, Sefika, and Adrian Paschke. "Retrieval-Augmented Generation-based Relation Extraction." arXiv preprint arXiv:2404.13397 (2024).