blt-entropy-patcher / bytelatent /data /iterators /multiprocess_iterator.py

Commit History

Let process start before yielding preloaded prefetch buffer, avoid needlessly losing buffer in edge cases (#75)
8f2cf88
unverified

par-meta commited on

Add approximate state persistence (#73)
ea1fc75
unverified

par-meta commited on

Update iterator inheritance, pass file format args, limit iterator (#63)
fc3399e
unverified

par-meta commited on

Fix multiprocessing dataloader checkpointing and use it in the train script (#50)
8c61ab5
unverified

par-meta commited on

This includes fixes that make checkpointing and reloading work correctly. (#35)
7044771
unverified

par-meta commited on

Initial commit
bcc039b

par-meta commited on