About the encoding type of tokenizer vocab
#1
by
isllmsmart - opened
Thanks for your work, but I can't find the right decoding type of the tokenizer vocab, e.g. there is no Chinese in it. I'm looking for your answer.
isllmsmart changed discussion status to
closed