Spaces:

lucalp
/

blt-entropy-patcher

Running on Zero

App Files Files Community

blt-entropy-patcher / bytelatent /data /iterators /arrow_iterator.py

Commit History

Cast int sample id to str (#96)

e299427
unverified

Srinivasan Iyer

sviyer commited on Apr 9

Reduce per file resources arrow uses (#77)

63913e4
unverified

par-meta commited on Mar 5

Correctly reset batch iterator at each arrow create_iter call. (#74)

c727844
unverified

par-meta commited on Mar 4

Update iterator inheritance, pass file format args, limit iterator (#63)

fc3399e
unverified

par-meta commited on Feb 22

Fix multiprocessing dataloader checkpointing and use it in the train script (#50)

8c61ab5
unverified

par-meta commited on Feb 13

Allow ArrowIterator to read from json (#45)

936d943
unverified

par-meta commited on Feb 6

Replace regular filesystem calls with fsspec + add s3 support (#18)

b0120da
unverified

par-meta commited on Jan 10

Initial commit

bcc039b

par-meta commited on Dec 12, 2024