Hugging Face
Models
Datasets
Spaces
Community
Docs
Enterprise
Pricing
Log In
Sign Up
Spaces:
yhavinga
/
dutch-tokenizer-arena
like
8
Running
App
Files
Files
Community
1
Fetching metadata from the HF Docker repository...
c78da21
dutch-tokenizer-arena
Ctrl+K
Ctrl+K
3 contributors
History:
79 commits
yhavinga
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
c78da21
over 1 year ago
css
add compression leaderboard
over 1 year ago
images
update
almost 2 years ago
js
add grok mixtral
over 1 year ago
patcher
add compression leaderboard
over 1 year ago
stats
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
over 1 year ago
utils
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
over 1 year ago
vocab
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
over 1 year ago
.gitattributes
Safe
2.32 kB
add grok mixtral
over 1 year ago
.gitignore
Safe
171 Bytes
update
almost 2 years ago
README.md
Safe
25.5 kB
Hack some Dutch tokenizers into it
over 1 year ago
app.py
Safe
565 Bytes
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
over 1 year ago
app_compression.py
Safe
4.75 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
over 1 year ago
app_playground.py
Safe
9.81 kB
update
over 1 year ago
config.py
Safe
1.72 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
over 1 year ago
examples.py
Safe
3.16 kB
add compression leaderboard
over 1 year ago
requirements.txt
Safe
96 Bytes
Pin protobuf dep
over 1 year ago
util.py
Safe
8.15 kB
Hack some Dutch tokenizers into it
over 1 year ago
