sudeshmu commited on
Commit
6c53489
·
verified ·
1 Parent(s): 861e577

Update README.md

Browse files

changed to fine_tune as its not mor

Files changed (1) hide show
  1. README.md +2 -16
README.md CHANGED
@@ -2,7 +2,6 @@
2
  license: mit
3
  library_name: transformers
4
  tags:
5
- - mixture-of-recursions
6
  - adaptive-computation
7
  - early-exiting
8
  - llama
@@ -17,26 +16,13 @@ pipeline_tag: text-generation
17
  model_type: llama
18
  ---
19
 
20
- # Mixture-of-Recursions (MoR): Learning Dynamic Recursive Depths for Adaptive Token-Level Computation
21
-
22
- <div align="center">
23
-
24
- [![Paper](https://img.shields.io/badge/Paper-arXiv:2507.10524-Green)](https://arxiv.org/abs/2507.10524)
25
- [![GitHub](https://img.shields.io/badge/GitHub-mixture_of_recursions-blue)](https://github.com/raymin0223/mixture_of_recursions)
26
- [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
27
 
28
  </div>
29
 
30
  ## Model Description
31
 
32
- This is a **Mixture-of-Recursions (MoR)** model that implements adaptive token-level computation through dynamic recursive depths. MoR addresses key bottlenecks in early-exiting techniques by introducing a unified framework that tackles both missing Key-Value (KV) cache problems and inefficient batched inference.
33
 
34
- **Key Features:**
35
- - 🚀 **Up to 2× greater inference throughput** compared to standard transformers at similar accuracy
36
- - 🧠 **Dynamic routing mechanism** that assigns optimal recursion depth to each token
37
- - 💾 **Recursion-wise KV caching strategy** that optimizes memory usage
38
- - ⚡ **Efficient batched inference** through parameter sharing
39
- - 🎯 **End-to-end trainable** architecture
40
 
41
  ### Model Details
42
 
@@ -64,7 +50,7 @@ import torch
64
  from transformers import AutoTokenizer, AutoModelForCausalLM
65
 
66
  # Load model and tokenizer
67
- model_name = "your-username/mixture-of-recursions-360m"
68
  tokenizer = AutoTokenizer.from_pretrained(model_name)
69
  model = AutoModelForCausalLM.from_pretrained(
70
  model_name,
 
2
  license: mit
3
  library_name: transformers
4
  tags:
 
5
  - adaptive-computation
6
  - early-exiting
7
  - llama
 
16
  model_type: llama
17
  ---
18
 
19
+ # Model Fine tunning on ineweb-edu-dedup, Hugging face open datasets
 
 
 
 
 
 
20
 
21
  </div>
22
 
23
  ## Model Description
24
 
 
25
 
 
 
 
 
 
 
26
 
27
  ### Model Details
28
 
 
50
  from transformers import AutoTokenizer, AutoModelForCausalLM
51
 
52
  # Load model and tokenizer
53
+ model_name = "your-username/fine_tune"
54
  tokenizer = AutoTokenizer.from_pretrained(model_name)
55
  model = AutoModelForCausalLM.from_pretrained(
56
  model_name,