Announcing the Common Pile and Comma v0.1
•
6
Common Pile
Enterprise
community
AI & ML interests
None defined yet.
Recent Activity
View all activity
Articles
Organization Card
The Common Pile
We are a group of researchers working together to collect and curate openly licensed and public domain data for training large language models. So far, we have released:
- The Common Pile v0.1, an 8 TB dataset of text from over 30 diverse sources
- Our paper: The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text
- Comma v0.1-1T and Comma v0.1-2T, 7B parameter LLMs trained on text from the Common Pile v0.1
- The training dataset used to train the Comma v0.1 models
- Our code for collecting data from each source
If you're interested in contributing, please open an issue on GitHub!
Collections
4
models
3
datasets
62
common-pile/comma_v0.1_training_dataset
Viewer
•
Updated
•
1.02B
•
5.49k
•
6
common-pile/youtube_filtered
Viewer
•
Updated
•
986k
•
91
•
1
common-pile/youtube
Viewer
•
Updated
•
1.13M
•
156
•
3
common-pile/wikiteam_filtered
Viewer
•
Updated
•
10.2M
•
99
common-pile/wikiteam
Viewer
•
Updated
•
552M
•
348
common-pile/wikimedia_filtered
Viewer
•
Updated
•
12.9M
•
105
•
1
common-pile/wikimedia
Viewer
•
Updated
•
78.1M
•
141
common-pile/uspto_filtered
Viewer
•
Updated
•
14.4M
•
661
common-pile/uspto
Viewer
•
Updated
•
16.2M
•
647
common-pile/usgpo_filtered
Viewer
•
Updated
•
2.34M
•
89