| Below is an improved version of the README in Markdown format. You can copy and paste the following text into your README file. | |
| --- | |
| # MasterControlAIML R1-Qwen2.5-1.5b SFT R1 JSON Unstructured-To-Structured LoRA Model | |
| [](https://github.com/unslothai/unsloth) | |
| This repository provides a fine-tuned Qwen2 model optimized for transforming unstructured text into structured JSON outputs according to a predefined schema. The model is finetuned from the base model **MasterControlAIML/DeepSeek-R1-Strategy-Qwen-2.5-1.5b-Unstructured-To-Structured** and leverages LoRA techniques for efficient adaptation. | |
| > **Key Highlights:** | |
| > | |
| > - **Developed by:** [bhaviktheslider](https://github.com/bhaviktheslider) | |
| > - **License:** [Apache-2.0](LICENSE) | |
| > - **Finetuned from:** `MasterControlAIML/DeepSeek-R1-Strategy-Qwen-2.5-1.5b-Unstructured-To-Structured` | |
| > - **Accelerated Training:** Achieved 2x faster training using [Unsloth](https://github.com/unslothai/unsloth) and Hugging Face's TRL library. | |
| --- | |
| ## Table of Contents | |
| - [Overview](#overview) | |
| - [Features](#features) | |
| - [Installation](#installation) | |
| - [Quick Start](#quick-start) | |
| - [Using Unsloth for Fast Inference](#using-unsloth-for-fast-inference) | |
| - [Using Transformers for Inference](#using-transformers-for-inference) | |
| - [Advanced Example with LangChain Prompt](#advanced-example-with-langchain-prompt) | |
| - [Contributing](#contributing) | |
| - [License](#license) | |
| - [Acknowledgments](#acknowledgments) | |
| --- | |
| ## Overview | |
| This model is tailored for tasks where mapping unstructured text (e.g., manuals, QA documents) into a structured JSON format is required. It supports hierarchical data extraction based on a given JSON Schema, ensuring that the generated outputs follow the exact structure and rules defined by the schema. | |
| --- | |
| ## Features | |
| - **Efficient Inference:** Utilizes the [Unsloth](https://github.com/unslothai/unsloth) library for fast model inference. | |
| - **Structured Output:** Maps text inputs into a strict JSON schema with hierarchical relationships. | |
| - **Flexible Integration:** Example code snippets show how to use both the Unsloth API and Hugging Face’s Transformers. | |
| - **Advanced Prompting:** Includes an example of using LangChain prompt templates for detailed instruction-driven output. | |
| --- | |
| ## Installation | |
| ### Prerequisites | |
| - **Python:** 3.8+ | |
| - **PyTorch:** (Preferably with CUDA support) | |
| - **Required Libraries:** `transformers`, `torch`, `unsloth`, `langchain` (for advanced usage) | |
| ### Installation Command | |
| Install the required Python packages with: | |
| ```bash | |
| pip install torch transformers unsloth langchain | |
| ``` | |
| --- | |
| ## Quick Start | |
| ### Using Unsloth for Fast Inference | |
| The Unsloth library allows you to quickly load and run inference with the model. Below is a basic example: | |
| ```python | |
| from unsloth import FastLanguageModel | |
| import torch | |
| # Specify the model name | |
| MODEL = "MasterControlAIML/R1-Qwen2.5-1.5b-SFT-R1-JSON-Unstructured-To-Structured-lora" | |
| # Load the model and tokenizer | |
| model, tokenizer = FastLanguageModel.from_pretrained( | |
| model_name=MODEL, | |
| max_seq_length=2048, | |
| dtype=None, | |
| load_in_4bit=False, | |
| ) | |
| # Prepare the model for inference | |
| FastLanguageModel.for_inference(model) | |
| # Define a prompt template | |
| ALPACA_PROMPT = """ | |
| Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. | |
| ### Instruction: | |
| {} | |
| ### Response: | |
| {} | |
| """ | |
| # Example: Create input and generate output | |
| instruction = "Provide a summary of the Quality Assurance Manual." | |
| prompt = ALPACA_PROMPT.format(instruction, "") | |
| inputs = tokenizer([prompt], return_tensors="pt").to("cuda") | |
| output = model.generate(**inputs, max_new_tokens=2000) | |
| # Decode and print the generated text | |
| print(tokenizer.batch_decode(output, skip_special_tokens=True)[0]) | |
| ``` | |
| --- | |
| ### Using Transformers for Inference | |
| If you prefer to use Hugging Face's Transformers directly, here’s an alternative example: | |
| ```python | |
| from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer | |
| import torch | |
| MODEL = "MasterControlAIML/R1-Qwen2.5-1.5b-SFT-R1-JSON-Unstructured-To-Structured-lora" | |
| # Initialize tokenizer and model | |
| tokenizer = AutoTokenizer.from_pretrained(MODEL) | |
| model = AutoModelForCausalLM.from_pretrained(MODEL, torch_dtype=torch.float16, device_map="auto") | |
| ALPACA_PROMPT = """ | |
| Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. | |
| ### Instruction: | |
| {} | |
| ### Response: | |
| {} | |
| """ | |
| # Define your text input | |
| TEXT = "Provide a detailed explanation of the QA processes in manufacturing." | |
| prompt = ALPACA_PROMPT.format(TEXT, "") | |
| inputs = tokenizer([prompt], return_tensors="pt").to("cuda") | |
| text_streamer = TextStreamer(tokenizer) | |
| # Generate output with specific generation parameters | |
| with torch.no_grad(): | |
| output_ids = model.generate( | |
| input_ids=inputs["input_ids"], | |
| attention_mask=inputs["attention_mask"], | |
| max_new_tokens=2000, | |
| temperature=0.7, | |
| top_p=0.9, | |
| repetition_penalty=1.1, | |
| streamer=text_streamer, | |
| pad_token_id=tokenizer.pad_token_id, | |
| ) | |
| # Print the decoded output | |
| print(tokenizer.decode(output_ids[0], skip_special_tokens=True)) | |
| ``` | |
| --- | |
| ### Advanced Example with LangChain Prompt | |
| For advanced users, the repository includes an example that integrates with LangChain to map hierarchical text data into a JSON schema. This example uses a prompt template to instruct the model on how to generate an output that includes both the JSON object (`<answer>`) and the reasoning behind the mapping decisions (`<think>`). | |
| ```python | |
| from langchain_core.prompts import PromptTemplate | |
| SYSTEM_PROMPT = """ | |
| ### Role: | |
| You are an expert data extractor specializing in mapping hierarchical text data into a given JSON Schema. | |
| ### DATA INPUT: | |
| - **Text:** ```{TEXT}``` | |
| - **Blank JSON Schema:** ```{SCHEMA}``` | |
| ### TASK REQUIREMENT: | |
| 1. Analyze the given text and map all relevant information strictly into the provided JSON Schema. | |
| 2. Provide your output in **two mandatory sections**: | |
| - **`<answer>`:** The filled JSON object | |
| - **`<think>`:** Reasoning for the mapping decisions | |
| ### OUTPUT STRUCTURE: | |
| ``` | |
| <think> /* Explanation of mapping logic */ </think> | |
| <answer> /* Completed JSON Object */ </answer> | |
| ``` | |
| ### STRICT RULES FOR GENERATING OUTPUT: | |
| 1. **Both Tags Required:** | |
| - Always provide both the `<think>` and `<answer>` sections. | |
| - If reasoning is minimal, state: "Direct mapping from text to schema." | |
| 2. **JSON Schema Mapping:** | |
| - Strictly map the text data to the given JSON Schema without modification or omissions. | |
| 3. **Hierarchy Preservation:** | |
| - Maintain proper parent-child relationships and follow the schema's hierarchical structure. | |
| 4. **Correct Mapping of Attributes:** | |
| - Map key attributes, including `id`, `idc`, `idx`, `level_type`, and `component_type`. | |
| 5. **JSON Format Compliance:** | |
| - Escape quotes, replace newlines with `\\n`, avoid trailing commas, and use double quotes exclusively. | |
| 6. **Step-by-Step Reasoning:** | |
| - Explain your reasoning within the `<think>` tag. | |
| ### IMPORTANT: | |
| If either the `<think>` or `<answer>` tags is missing, the response will be considered incomplete. | |
| """ | |
| # Create a prompt template with LangChain | |
| system_prompt_template = PromptTemplate(template=SYSTEM_PROMPT, input_variables=["TEXT", "SCHEMA"]) | |
| # Format the prompt with your text and JSON schema | |
| system_prompt_str = system_prompt_template.format( | |
| TEXT="Your detailed text input here...", | |
| SCHEMA="""{ | |
| "type": "object", | |
| "properties": { | |
| "id": {"type": "string", "description": "Unique identifier."}, | |
| "title": {"type": "string", "description": "Section title."}, | |
| "level": {"type": "integer", "description": "Hierarchy level."}, | |
| "level_type": {"type": "string", "enum": ["ROOT", "SECTION", "SUBSECTION", "DETAIL_N"], "description": "Hierarchy type."}, | |
| "component": { | |
| "type": "array", | |
| "items": { | |
| "type": "object", | |
| "properties": { | |
| "idc": {"type": "integer", "description": "Component ID."}, | |
| "component_type": {"type": "string", "enum": ["PARAGRAPH", "TABLE", "CALCULATION", "CHECKBOX"], "description": "Component type."}, | |
| "metadata": {"type": "string", "description": "Additional metadata."}, | |
| "properties": {"type": "object"} | |
| }, | |
| "required": ["idc", "component_type", "metadata", "properties"] | |
| } | |
| }, | |
| "children": {"type": "array", "items": {}} | |
| }, | |
| "required": ["id", "title", "level", "level_type", "component", "children"] | |
| }""" | |
| ) | |
| # Use the system prompt with your inference code as shown in previous examples. | |
| ``` | |
| --- | |
| ## Contributing | |
| Contributions, bug reports, and feature requests are welcome! Please open an issue or submit a pull request if you would like to contribute to this project. | |
| --- | |
| ## License | |
| This project is licensed under the [Apache-2.0 License](LICENSE). | |
| --- | |
| ## Acknowledgments | |
| - **Unsloth:** For providing fast model inference capabilities. ([GitHub](https://github.com/unslothai/unsloth)) | |
| - **Hugging Face:** For the [Transformers](https://github.com/huggingface/transformers) and [TRL](https://github.com/huggingface/trl) libraries. | |
| - **LangChain:** For advanced prompt management and integration. | |
| - And, of course, thanks to the community and contributors who helped shape this project. | |
| --- | |
| Enjoy using the model, and happy coding! |