legacy-datasets/wikipedia
Updated • 100k • 662
How to use mtreviso/ct5-small-en-wiki-l2r with Transformers:
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("mtreviso/ct5-small-en-wiki-l2r")
model = AutoModelForSeq2SeqLM.from_pretrained("mtreviso/ct5-small-en-wiki-l2r", device_map="auto")Github: https://github.com/mtreviso/chunked-t5
This is a variant of cT5 that was trained with a left-to-right autoregressive decoding mask. As a consequence, it does not support parallel decoding, but it still predicts the end-of-chunk token </c> at the end of each chunk.