pile-t5-base / README.md
lintang's picture
Update README.md
909902f verified
|
raw
history blame
2.45 kB
metadata
datasets:
  - EleutherAI/pile
language:
  - en
pipeline_tag: text2text-generation
tags:
  - summarization
  - translation

Model Card for T5v2 Base

Table of Contents

  1. Model Details
  2. Uses
  3. Bias, Risks, and Limitations
  4. Training Details
  5. Evaluation
  6. Environmental Impact
  7. Citation
  8. Model Card Authors
  9. How To Get Started With the Model

Model Details

Model Description

More information needed.

Uses

Direct Use and Downstream Use

More information needed.

Out-of-Scope Use

More information needed.

Bias, Risks, and Limitations

More information needed.

Recommendations

More information needed.

Training Details

Training Data

The model was pre-trained on the Pile using an unsupervised denoising objective,

Training Procedure

More information needed.

Evaluation

Testing Data, Factors & Metrics

More information needed.

Results

More information needed.

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

  • Hardware Type: Google Cloud TPU Pods
  • Hours used: More information needed
  • Cloud Provider: GCP
  • Compute Region: More information needed
  • Carbon Emitted: More information needed

Citation

BibTeX:

@article{2024t5v2,
  author  = {Lintang Sutawika and Aran Komatsuzaki and Colin Raffel},
  title   = {T5v2, an update of T5},
  year    = {2024},
  url     = {}
}

How to Get Started with the Model

Use the code below to get started with the model.

Click to expand
from transformers import UMT5Tokenizer, UMT5Model

tokenizer = UMT5Tokenizer.from_pretrained("EleutherAI/t5-v2-base")
model = UMT5Model.from_pretrained("EleutherAI/t5-v2-base")

input_ids = tokenizer(
    "Studies have been shown that owning a dog is good for you", return_tensors="pt"
).input_ids  # Batch size 1
decoder_input_ids = tokenizer("Studies show that", return_tensors="pt").input_ids  # Batch size 1

# forward pass
outputs = model(input_ids=input_ids, decoder_input_ids=decoder_input_ids)
last_hidden_states = outputs.last_hidden_state