batteryphil/besticandofornow

Operating-Organism Mamba-2 370M (AdamW Stabilized)

This repository contains the fine-tuned Mamba-2 370M model, explicitly trained on the Operating-Organism codebase and context. The model has been stabilized using standard AdamW (moving away from the structural collapse caused by the Continuous Vote Descent optimizer) and is fully ready for both Python inference and UEFI bare-metal execution.

Files Provided

1. mono250m_sft_live.pt: The PyTorch state dictionary (Continuous bfloat16 AdamW weights).
2. mono250m_sft_live_bf16.bin: The 1GB native bfloat16 continuous payload tailored for bare-metal UEFI loading.

1. How to run in Python

Because this model was derived from state-spaces/mamba2-370m, you must load the base architecture first and inject these weights.

import torch
from transformers import AutoTokenizer
from mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel
from huggingface_hub import hf_hub_download

# 1. Load Tokenizer & Base Architecture
tokenizer = AutoTokenizer.from_pretrained("EleutherAI/gpt-neox-20b")
model = MambaLMHeadModel.from_pretrained("state-spaces/mamba2-370m", dtype=torch.bfloat16, device="cuda")

# 2. Download and Inject the Custom Weights
ckpt_path = hf_hub_download(repo_id="batteryphil/besticandofornow", filename="mono250m_sft_live.pt")
ckpt = torch.load(ckpt_path, map_location='cuda', weights_only=False)
model.load_state_dict(ckpt['state_dict'], strict=True)
model.eval()

# 3. Generate!
prompt = "User: Explain the purpose of the Operating-Organism kernel.\n\nAssistant:"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")

with torch.no_grad():
    outputs = model.generate(
        input_ids=inputs.input_ids,
        max_length=128,
        temperature=0.7,
        top_p=0.9,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. How to run on UEFI Bare-Metal (QEMU/KVM)

To deploy this in the Operating-Organism ecosystem, you will use the .bin file which is formatted for zero-copy high-memory loading (0x100000000ULL).

1. Download mono250m_sft_live_bf16.bin:

wget https://huggingface.co/batteryphil/besticandofornow/resolve/main/mono250m_sft_live_bf16.bin

2. Split the model into UEFI chunks (within your llm-baremetal directory):

dd if=mono250m_sft_live_bf16.bin of=model_shard0.bin bs=1M count=1024 status=progress
dd if=mono250m_sft_live_bf16.bin of=model_shard1.bin bs=1M skip=1024 count=1024 status=progress
dd if=mono250m_sft_live_bf16.bin of=model_shard2.bin bs=1M skip=2048 status=progress

3. Build the FAT32 UEFI Boot Image:

./make_image.sh

4. Boot the OS!

./run_kvm.sh

Architecture Notes

This model successfully utilizes the AVX2 bfloat16 instructions in the UEFI environment, bypassing the need for discrete PRIME quantization grids.