jialicheng/samsum_t5-xl

samsum_42

This model is a fine-tuned version of google/t5-v1_1-xl on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.1251
• Rouge1: 54.9849
• Rouge2: 30.5117
• Rougel: 45.8377
• Rougelsum: 50.8325
• Gen Len: 26.2286

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	230	1.2376	51.063	26.1483	42.2509	47.1434	25.0954
No log	2.0	460	1.1684	53.4831	28.8893	44.3189	49.612	27.4010
No log	3.0	690	1.1291	54.4367	29.8842	45.5296	50.4519	26.2103
No log	4.0	921	1.1242	54.71	30.5588	45.4993	50.735	26.8154
1.5611	4.99	1150	1.1251	54.9849	30.5117	45.8377	50.8325	26.2286

Framework versions

• Transformers 4.39.3
• Pytorch 2.3.0+cu121
• Datasets 2.19.0
• Tokenizers 0.15.2