Hugging Face's logo
Pro Creations's picture
4 5 10

Pro Creations PRO

ProCreations
Eileen234's profile picture Drnerdy81's profile picture emredeveloper's profile picture
·
https://pro-ai.glitch.me

AI & ML interests

AGI and small scale-high quality AI

Recent Activity

replied to Ruurd's post about 5 hours ago
The past year I have been trying to get diffusion models to work for language generation, without having to retrain a LLM from scratch. And recently, we finally succeeded: We introduce "LAD: LoRA-Adapted Denoiser", a method to convert a LLaMA model into a text diffusion model using LoRA finetuning and structured input corruption. Unlike autoregressive models, diffusion-style generation iteratively refines a noised sequence. However, most current diffusion approaches rely on all-parameter re-training and repeatedly remasking tokens, which is costly and slow during both training and inference! 🧠 With LAD: - We can finetune an autoregressive model for diffusive generation in just 10 hours on a single GPU. - Test-time compute is fully adjustable: fewer steps means faster outputs while more steps improve output quality. - Due to our unique noising schedule, remasking is not always needed during inference. All tokens are attended to in each iteration! 🔍 LAD is built using: – A frozen LLaMA-8B backbone – Structured noising: token swaps, duplications, replacements, span shifts – Modified attention masks for bidirectional decoding 💡 We show that even small, fast-trained models can perform diffusive generation — with competitive benchmark performance, perplexity and more flexible test-time behavior than traditional transformers. 🎯 Try the demo and read the write-up here: https://ruurdkuiper.github.io/tini-lad/
liked a dataset about 9 hours ago
ProCreations/Ultra-FineWeb-EDU
posted an update about 23 hours ago
NEW DATASETTTTT https://huggingface.co/datasets/ProCreations/Ultra-FineWeb-EDU This dataset is an extremely filtered version of https://huggingface.co/datasets/openbmb/Ultra-FineWeb for educational content only, inspired by https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu Right now dataset size is small (64k examples) and more examples are being processed as you read this!
View all activity

Organizations

Hugging Face Discord Community's profile picture ProCreations Devoloper Team's profile picture

Posts 29

View all Posts

Collections 1