Update ppl evals to work with blt model, in addition to entropy model (#82) 083656c unverified par-meta commited on Mar 13
Pass mask in packing_iterator, correctly handle last batch, fix masking (#65) 08b8c7c unverified par-meta commited on Feb 27
Remove byte tokenizer and add config args to switch between byte/patch packing (#68) aeb95f1 unverified par-meta commited on Feb 25
Update iterator inheritance, pass file format args, limit iterator (#63) fc3399e unverified par-meta commited on Feb 22
Initial codes and scripts for training entropy model (#34) 7622d28 unverified par-meta commited on Jan 27
