Post
242
Is it time to start developing sparse attention again?
https://github.com/SmallDoges/flash-sparse-attention
https://github.com/SmallDoges/flash-sparse-attention
Loser Cheems
JingzeShi
AI & ML interests
I like training small languge models.
Recent Activity
updated
a model
6 days ago
DIAL-TFM/TFM-tokenizer
published
a model
6 days ago
DIAL-TFM/TFM-tokenizer
updated
a dataset
8 days ago
JingzeShi/table-sample
Organizations
Articles 1
Article
7
Trainable Dynamic Mask Sparse Attention: Bridging Efficiency and Effectiveness in Long-Context Language Models
models
7
JingzeShi/OpenSeek-1.4B-A0.4B-KTO
Text Generation
•
1B
•
Updated
•
8
JingzeShi/OpenSeek-1.4B-A0.4B
Text Generation
•
1B
•
Updated
•
7
JingzeShi/Doge-20M
Text Generation
•
37.6M
•
Updated
•
19
JingzeShi/Doge-320M-Reason-checkpoint
0.4B
•
Updated
•
5
JingzeShi/Doge-320M-Reason-Distill
Text Generation
•
0.3B
•
Updated
•
5
JingzeShi/Doge-120M-MoE
0.1B
•
Updated
•
8
JingzeShi/Mixtral-7B-v0.1
Text Generation
•
Updated
•
5
datasets
19
JingzeShi/table-sample
Viewer
•
Updated
•
2
•
15
JingzeShi/OpenSeek-Pretrain-100B
Viewer
•
Updated
•
56.1M
•
488
JingzeShi/minervamath
Viewer
•
Updated
•
272
•
23
JingzeShi/amc23
Viewer
•
Updated
•
40
•
9
JingzeShi/gpqa
Viewer
•
Updated
•
198
•
35
JingzeShi/math_500
Viewer
•
Updated
•
500
•
24
JingzeShi/aime25
Viewer
•
Updated
•
30
•
18
JingzeShi/aime24
Viewer
•
Updated
•
30
•
16
JingzeShi/aime24-try-run
Viewer
•
Updated
•
2
•
23
JingzeShi/test_dataset
Viewer
•
Updated
•
1
•
31