README.md

metadata

license: llama3.2
license_link: https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/blob/main/LICENSE.txt
library: llama.cpp
library_link: https://github.com/ggerganov/llama.cpp
base_model:
  - meta-llama/Llama-3.2-1B-Instruct
language:
  - en
  - de
  - fr
  - it
  - pt
  - hi
  - es
  - th
pipeline_tag: text-generation
tags:
  - nlp
  - code
  - gguf

LLaMA 3.2 1B Instruct

1. Model Title

• Name: LLaMA 3.2 1B Instruct
• Parameter Size: 1B (1.23B)

2. Quantization Information

• Available Formats:
  • ggml-model-q8_0.gguf: 8-bit quantization for resource efficiency and good performance.
  • ggml-model-f16.gguf: Half-precision (16-bit) floating-point format for enhanced precision.
• Quantization Library: llama.cpp
• Use Cases: Recommended for tasks such as multilingual dialogue, text generation, and summarization.

3. Model Brief

LLaMA 3.2 1B Instruct is a multilingual instruction-tuned language model, optimized for various dialogue tasks. It has been trained on a diverse set of publicly available data and performs well on common NLP benchmarks. The model architecture leverages improved transformer optimizations, making it effective for both text-only and code tasks.

• Purpose: Multilingual dialogue generation and summarization.
• Model Family: LLaMA 3.2
• Architecture: Auto-regressive Transformer with Grouped-Query Attention (GQA)
• Training Data: A mix of publicly available multilingual data, covering up to 9T tokens.
• Supported Languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
• Release Date: September 25, 2024
• Context Length: 128k tokens
• Knowledge Cutoff: December 2023

4. Core Library Information

• Library: llama.cpp
  • Repository Link
• Model Base: meta-llama/Llama-3.2-1B-Instruct

5. Safety and Responsible Use

LLaMA 3.2 1B is designed with safety in mind but still carries inherent risks due to its generative nature. It may produce biased, harmful, or unpredictable responses, especially for less-tested languages or sensitive prompts.

• Testing and Risk Assessment: Initial testing has focused on English outputs, and coverage for other languages is ongoing.
• Limitations: As with most LLMs, LLaMA 3.2 may not fully adhere to user instructions or safety guidelines and might exhibit unexpected behavior.
• Responsible Use Guidelines: For deployment, thorough testing is advised to align outputs with application-specific safety requirements. Refer to the Responsible Use Guide for more details.