Text Generation
Transformers
PyTorch
English
llama
causal-lm
llama2
fine-tuning
wikipedia
text-generation-inference
Instructions to use unionai/Llama-2-7b-hf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use unionai/Llama-2-7b-hf with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="unionai/Llama-2-7b-hf")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("unionai/Llama-2-7b-hf") model = AutoModelForCausalLM.from_pretrained("unionai/Llama-2-7b-hf") - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use unionai/Llama-2-7b-hf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "unionai/Llama-2-7b-hf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unionai/Llama-2-7b-hf", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/unionai/Llama-2-7b-hf
- SGLang
How to use unionai/Llama-2-7b-hf with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "unionai/Llama-2-7b-hf" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unionai/Llama-2-7b-hf", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "unionai/Llama-2-7b-hf" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "unionai/Llama-2-7b-hf", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use unionai/Llama-2-7b-hf with Docker Model Runner:
docker model run hf.co/unionai/Llama-2-7b-hf
Ctrl+K
- flyte11g9iwbg
- flyte19klvulo
- flyte24da_57k
- flyte5to6vtxz
- flyte6a_2j95z
- flyte7htre9gj
- flyte9y8545zr
- flyteeopmp26r
- flyteet3ed7rr
- flyteg7cwhdqx
- flytegwz7gead
- flyteh3d_ydsb
- flytehp3ce2w5
- flytek6j4vh44
- flyteknfii7qb
- flytem4no92qv
- flytemk5qri4f
- flytep9efi7h3
- flytepxngzev1
- flyteraquk0cj
- flyterpqo54fv
- flytex31ghl6k
- flyteyao8jgm7
- flyteyfv3rs04
- tmp1irzl5w5
- tmp2uwb6tgl
- tmp5210xtp5
- tmp5_j_c0zj
- tmp9bh3sdsi
- tmp_l9cweyv
- tmpbq_uwzvs
- tmpcwd32mn0
- tmpdouqave3
- tmpefjtsdm5
- tmpftz082d8
- tmphrsaxah2
- tmpi_fed6hf
- tmpj0u3x6ea
- tmpn7s2kko7
- tmpnmkfim5e
- tmpr9sct6r4
- tmprlrkxdh2
- tmpscdch5ch
- tmptwgnkwb4
- tmpub30lm_f
- 4.02 kB
- 169 Bytes
- 21 Bytes
- 609 Bytes
- 0 Bytes