metadata
license: apache-2.0
base_model: Salesforce/codet5-small
tags:
- generated_from_trainer
model-index:
- name: training
results: []
training
This model is a fine-tuned version of Salesforce/codet5-small on my own dataset.
Model description
Generates descriptions of git commits which have code smells which possibly signify technical debt.
Intended uses & limitations
Use with caution. Limited by small training set and limited variety of training set labels. Improvements in progress.
Training procedure
one epoch of training on the dataset referred to above
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 1
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 100
- total_train_batch_size: 100
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
- mixed_precision_training: Native AMP
Framework versions
- Transformers 4.35.0
- Pytorch 2.1.0+cu118
- Datasets 2.14.6
- Tokenizers 0.14.1