metadata

title: FlexChunk SpMV Demo
emoji: ⚡
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 4.39.0
app_file: app.py
pinned: false

FlexChunk: Enabling 100M×100M Out-of-Core SpMV

This Hugging Face Space demonstrates FlexChunk, an algorithm for performing Sparse Matrix-Vector Multiplication (SpMV) on matrices potentially too large to fit in RAM.

Key Idea: Divide the matrix into manageable horizontal chunks, process them sequentially, and use minimal memory (~1.7 GB for 100M×100M SpMV).

Interactive Demo

The app above allows you to:

Generate a sparse matrix (adjust Size, Density, Chunks, Challenging flag).
Run SpMV using FlexChunk (chunking to disk) and SciPy (emulating disk load).
Compare performance (timings) and correctness.

Note: Larger matrices/densities will take longer. Limits are set for public infrastructure.

Performance Highlights

FlexChunk demonstrates near-linear scaling in time and memory for increasing matrix dimensions.

Time Performance and Memory Usage: See the original article for detailed performance graphs and analysis.

Benchmark Results

Selected results showing performance on large matrices:

Matrix Size	Non-zero Elements	Total Time	Peak RAM Usage
...	...	...	...
1.0M × 1.0M	1.2M	1.07 s	17.00 MB
10.0M × 10.0M	12.0M	10.21 s	170.00 MB
30.0M × 30.0M	36.0M	31.13 s	510.00 MB
50.0M × 50.0M	62.5M	55.27 s	850.00 MB
70.0M × 70.0M	88.2M	1 min 17.1 s	1.19 GB
100.0M × 100.0M	120.0M	1 min 47.1 s	1.70 GB

*(Full table in the original README - Update this link!)*

More Information

See the original post for technical details: FlexChunk: Enabling 100M×100M Out-of-Core SpMV