Upload 10 files

citation_generation_lora/README.md

@@ -0,0 +1,187 @@
+---
+license: apache-2.0
+language:
+- en
+pipeline_tag: text-generation
+base_model: ibm-granite/granite-3.3-8b-instruct
+library_name: peft
+library_name: transformers
+---
+
+# LoRA Adapter for Citation Generation
+Welcome to Granite Experiments!
+
+Think of Experiments as a preview of what's to come. These projects are still under development, but we wanted to let the open-source community take them for spin! Use them, break them, and help us build what's next for Granite – we'll keep an eye out for feedback and questions. Happy exploring!
+
+Just a heads-up: Experiments are forever evolving, so we can't commit to ongoing support or guarantee performance.
+
+# Model Summary 
+
+This is a RAG-specific LoRA adapter for [ibm-granite/granite-3.3-8b-instruct](https://huggingface.co/ibm-granite/granite-3.3-8b-instruct) that is fine-tuned for the citation generation task. Given a multi-turn conversation between a user and an AI assistant ending with an assistant response and a set of documents/passages on which the last assistant response is supposed to be based, the adapter generates citations for the last assistant response from the provided documents/passages. The LoRA adapter has the following features:
+1. **Fine-grained citations:** The adapter generates citations for each sentence in the assistant response (when available). Moreover, each citation consists of a set of sentences from the documents/passages that support the corresponding sentence in the assistant response.   
+2. **Post-hoc citation generation:** Since the adapter takes the assistant response as input, it can generate citations for responses generated by any LLM. Pick your favorite LLM and use the adapter to generate post-hoc citations!
+
+</br>
+
+- **Developer:** IBM Research
+- **Model type:** LoRA adapter for [ibm-granite/granite-3.3-8b-instruct](https://huggingface.co/ibm-granite/granite-3.3-8b-instruct)
+- **License:** [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0)
+
+## Intended use
+This is a LoRA adapter that gives the ability to generate citations for the last assistant response in a multi-turn RAG conversation based on a set of provided documents/passages. It can be used to generate post-hoc citations for assistant responses generated by any LLM in a RAG setting.  
+
+> [!TIP]  
+> Note: While you can invoke the adapter directly, as outlined below, we highly recommend calling it through [granite-io](https://github.com/ibm-granite/granite-io), which wraps the model with a tailored I/O processor, enabling a friendlier development interface. The I/O processor takes care of several data transformation/validation tasks that would be otherwise required (incl. splitting the input documents and assistant response into sentences before calling the adapter as well as validating the adapter's output and transforming the returned sentence IDs into spans over the documents and the response).
+
+**Model input**: The input to the model is conceptually a list of conversational turns ending with an assistant response and a list of documents converted to a string using the `apply_chat_template` function. For the adapter to work, the last assistant response as well as the documents should be pre-split into sentences. In more detail, the primary inputs are the following three items, each represented in JSON:  
+
+- **conversation**: A list of conversational turns between the user and the assistant, where each item in the list is a dictionary with fields `role` and `content`. The `role` equals to either `user` or `assistant`, denoting user and assistant turns, respectively, while the `content` field contains the corresponding user/assistant utterance. The conversation should end with an assistant turn and the `text` field of that turn should contain the assistant utterance with each sentence prefixed with a response sentence ID of the form `<rI>`, where `I` is an integer. The numbering should start from 0 (for the first sentence) and be incremented by one for each subsequent sentence in the last assistant turn. Note that only the last assistant turn should be split into sentences as described above; earlier assistant turns (as well as all user turns) should be maintained in their original form.
+- **instruction**: A task instruction, which is encoded as a dictionary with fields `role` and `content`, where `role` equals to `system` and `content` equals to the following string describing the citation generation task: `Split the last assistant response into individual sentences. For each sentence in the response, identify the statement IDs from the documents that it references. Ensure that your output includes all response sentence IDs, and for each response sentence ID, provide the corresponding referring document sentence IDs.` 
+- **documents**: A list of documents, where each item in the list is a dictionary with fields `doc_id` and `text`. The `text` field contains the text of the corresponding document with each sentence prefixed with a context sentence ID of the form `<cI>`, where `I` is an integer. The context sentence ID numbers should start from 0 (for the first sentence of the first document) and be incremented by one for each subsequent sentence. The numbers should continue to be incremented across documents to ensure that each context sentence ID appears once across the entire list of documents. For instance, if the last sentence of the 1st document has context sentence ID `<cn>`, then the first sentence of the 2nd document is expected to have ID `<cn+1>`.  
+
+To prompt the LoRA adapter, we combine the above components as follows: We first append the **instruction** to the end of the **conversation** to generate an **augmented_conversation** list. Then we invoke the `apply_chat_template` function with parameters: conversation = **augmented_conversation** and documents = **documents**.
+
+**Model output**: When prompted with the above input, the model generates the citations for each sentence of the last assistant response in the form of a JSON array. The array is of the form `[{"r": 0, "c": [...]}, {"r": 1, "c": [...]}, ...}]`, where a JSON object of the form `{"r": k, "c": [l, m]}`, where `k, l, m` integers, denotes that the response sentence with ID `<rk>` is supported by the context sentences with IDs `<cl>` and `<cm>`.   
+
+
+## Quickstart Example
+
+As explained above, it is highly recommended to use the LoRA adapter through [granite-io](https://github.com/ibm-granite/granite-io).
+
+However, if you prefer to invoke the LoRA adapter directly, you can use the following code. Note that the code assumes that the documents and the last assistant response have been already split into sentences. 
+
+```
+import torch
+from transformers import AutoTokenizer,  AutoModelForCausalLM
+from peft import PeftModel, PeftConfig
+import json
+
+BASE_NAME = "ibm-granite/granite-3.3-8b-instruct"
+LORA_NAME = "ibm-granite/granite-3.3-8b-lora-rag-citation-generation"  
+device=torch.device('cuda' if torch.cuda.is_available() else 'cpu')
+
+tokenizer = AutoTokenizer.from_pretrained(BASE_NAME, padding_side='left', trust_remote_code=True)
+model_base = AutoModelForCausalLM.from_pretrained(BASE_NAME, device_map="auto")
+model_citation = PeftModel.from_pretrained(model_base, LORA_NAME)
+
+conversation = [
+    {"role": "user", "content": "What is the visibility level of Git Repos and Issue Tracking projects?"}, 
+    {"role": "assistant", "content": "<r0> Git Repos and Issue Tracking projects can have one of the following visibility levels: private, internal, or public. <r1> Private projects are visible only to project members, internal projects are visible to all users that are logged in to IBM Cloud, and public projects are visible to anyone. <r2> By default, new projects are set to private visibility level, which is the most secure for your data."}]
+
+documents = [
+    {"doc_id": 0, "text": "<c0> Git Repos and Issue Tracking is an IBM-hosted component of the Continuous Delivery service. <c1> All of the data that you provide to Git Repos and Issue Tracking, including but not limited to source files, issues, pull requests, and project configuration properties, is managed securely within Continuous Delivery. <c2> However, Git Repos and Issue Tracking supports various mechanisms for exporting, sending, or otherwise sharing data to users and third parties. <c3> The ability of Git Repos and Issue Tracking to share information is typical of many social coding platforms. <c4> However, such sharing might conflict with regulatory controls that apply to your business. <c5> After you create a project in Git Repos and Issue Tracking, but before you entrust any files, issues, records, or other data with the project, review the project settings and change any settings that you deem necessary to protect your data. <c6> Settings to review include visibility levels, email notifications, integrations, web hooks, access tokens, deploy tokens, and deploy keys. <c7> Project visibility levels \n\nGit Repos and Issue Tracking projects can have one of the following visibility levels: private, internal, or public. <c8> * Private projects are visible only to project members. <c9> This setting is the default visibility level for new projects, and is the most secure visibility level for your data. <c10> * Internal projects are visible to all users that are logged in to IBM Cloud. <c11> * Public projects are visible to anyone. <c12> To limit project access to only project members, complete the following steps:\n\n\n\n1. <c13> From the project sidebar, click Settings > General. <c14> 2. <c15> On the General Settings page, click Visibility > project features > permissions. <c16> 3. <c17> Locate the Project visibility setting. <c18> 4. <c19> Select Private, if it is not already selected. <c20> 5. <c21> Click Save changes. <c22> Project membership \n\nGit Repos and Issue Tracking is a cloud hosted social coding environment that is available to all Continuous Delivery users. <c23> If you are a Git Repos and Issue Tracking project Maintainer or Owner, you can invite any user and group members to the project. <c24> IBM Cloud places no restrictions on who you can invite to a project."},
+    {"doc_id": 1, "text": "<c25> After you create a project in Git Repos and Issue Tracking, but before you entrust any files, issues, records, or other data with the project, review the project settings and change any settings that are necessary to protect your data. <c26> Settings to review include visibility levels, email notifications, integrations, web hooks, access tokens, deploy tokens, and deploy keys. <c27> Project visibility levels \n\nGit Repos and Issue Tracking projects can have one of the following visibility levels: private, internal, or public. <c28> * Private projects are visible only to project members. <c29> This setting is the default visibility level for new projects, and is the most secure visibility level for your data. <c30> * Internal projects are visible to all users that are logged in to IBM Cloud. <c31> * Public projects are visible to anyone. <c32> To limit project access to only project members, complete the following steps:\n\n\n\n1. <c33> From the project sidebar, click Settings > General. <c34> 2. <c35> On the General Settings page, click Visibility > project features > permissions. <c36> 3. <c37> Locate the Project visibility setting. <c38> 4. <c39> Select Private, if it is not already selected. <c40> 5. <c41> Click Save changes. <c42> Project email settings \n\nBy default, Git Repos and Issue Tracking notifies project members by way of email about project activities. <c43> These emails typically include customer-owned data that was provided to Git Repos and Issue Tracking by users. <c44> For example, if a user posts a comment to an issue, Git Repos and Issue Tracking sends an email to all subscribers. <c45> The email includes information such as a copy of the comment, the user who posted it, and when the comment was posted. <c46> To turn off all email notifications for your project, complete the following steps:\n\n\n\n1. <c47> From the project sidebar, click Settings > General. <c48> 2. <c49> On the **General Settings **page, click Visibility > project features > permissions. <c50> 3. <c51> Select the Disable email notifications checkbox. <c52> 4. <c53> Click Save changes. <c54> Project integrations and webhooks"}]
+
+# Add system prompt
+citation_sys_prompt = "Split the last assistant response into individual sentences. For each sentence in the response, identify the statement IDs from the documents that it references. Ensure that your output includes all response sentence IDs, and for each response sentence ID, provide the list of corresponding referring document sentence IDs. The output must be a json structure."
+conversation.append({"role": "system", "content": citation_sys_prompt})
+
+# Generate answer
+input_text = tokenizer.apply_chat_template(conversation=conversation, documents=documents, tokenize=False)
+inputs = tokenizer(input_text, return_tensors="pt")
+output = model_citation.generate(inputs["input_ids"].to(device), attention_mask=inputs["attention_mask"].to(device), max_new_tokens=500)
+output_text = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
+print("Output: ")
+print(json.loads(output_text))
+```
+
+
+## Training Details
+
+The LoRA adapter was trained on synthetically-generated citation datasets. The process of generating the training data consisted of two main steps:
+- **Multi-turn RAG conversation generation:** Starting from publicly available document corpora, we generated a set of multi-turn RAG data, consisting of multi-turn conversations grounded on passages retrieved from the corpora. For details on the RAG conversation generation process please refer to the [Granite Technical Report](https://github.com/ibm-granite/granite-3.0-language-models/blob/main/paper.pdf) and [Lee, Young-Suk, et al.](https://arxiv.org/pdf/2409.11500).
+- **Citation generation:** For each turn of the multi-turn RAG conversations from the previous step, we used a multi-step synthetic citation generation pipeline to generate citations for the assistant response.
+
+The resulting data instances were used to train the LoRA adapter.
+
+### Training Data
+
+The following public datasets were used as seed datasets for the multi-turn RAG conversation generation process:
+- [CoQA](https://stanfordnlp.github.io/coqa/) - Wikipedia passages
+- [MultiDoc2Dial](https://huggingface.co/datasets/IBM/multidoc2dial)
+- [QuAC](https://huggingface.co/datasets/allenai/quac)
+
+#### Training Hyperparameters
+
+The LoRA adapter was fine-tuned using PEFT under the following regime: rank = 8, learning rate = 1e-5, and 90/10 split between training and validation.
+
+
+## Evaluation
+
+We evaluate the LoRA adapter on two citation benchmarks:
+- [ALCE](https://aclanthology.org/2023.emnlp-main.398/): Evaluates the ability of models to produce document/passage-level citations (i.e., identify the documents/passages that support a statement in the response).
+- [LongBench-Cite](https://arxiv.org/abs/2409.02897): Evaluates the ability of models to produce fine-grained span-level citations (i.e., identify the spans within the input documents/passages that support a statement in the response) with a focus on long contexts.
+
+Since the LoRA adapter is a post-hoc citation generation approach, its performance on the two benchmarks depends on the assistant responses for which it is asked to generate citations. To facilitate an apples-to-apples comparison, for each experiment, we keep the assistant responses the same and change the model that is used to generate the citations. In particular, we prompt an LLM to create an assistant response together with citations and evaluate the generated citations on the corresponding benchmark. Then, we compute and evaluate the citations generated for the same LLM response by the LoRA adapter.    
+
+### Evaluation on ALCE
+
+For the ALCE evaluation, we prompt Llama-3.1-70B-Instruct and Mixtral-8x22B-Instruct to generate both the assistant response and corresponding passage-level citations. We first calculate the performance of the citations generated by these models on ALCE. Subsequently, we feed the responses of these models (leaving out the citations) to the LoRA adapter and evaluate its generated citations. The results are shown in the table below: 
+
+Model used to generate response | Model used to generate citations                      | Recall          | Precision         |  F1       |
+|--------------| ----------------------------- | --------------- | ----------------- | --------- |
+| Llama-3.1-70B-Instruct | Llama-3.1-70B-Instruct        | 61.4            | 58.1              | 59.7      |
+| Llama-3.1-70B-Instruct | Granite-3.3-8B LoRA citations | 55.4            | 65.0              | 59.8      |
+| Mixtral-8x22B-Instruct | Mixtral-8x22B-Instruct        | 62.2            | 62.5              | 62.3      |
+| Mixtral-8x22B-Instruct | Granite-3.3-8B LoRA citations | 55.6            | 69.0              | 61.6      |
+
+We observe that the LoRA adapter performs on par with much bigger models when those are prompted to create passage-level citations. It is interesting to note that while the adapter's F1 performance is similar to the baselines, it exhibits a different precision-recall trade-off, trading lower recall for higher precision.   
+
+Notes:
+- All results are reported on the ELI5 dataset using the ORACLE (5-psg) setting.
+- To prompt Llama and Mixtral, we employ a setting similar to the one proposed in the ALCE paper; in particular we use a two-shot prompt comprised of two of the ICL examples from ALCE as well as a slightly modified version of the instruction from the paper.
+- Sentence splitting of context/response is performed using NLTK.
+- Finally, since ALCE expects passage-level citations, we elevate the finer-grained citations produced by the LoRA adapter to the passage level before running the ALCE evaluation.
+
+
+### Evaluation on LongBench-Cite
+
+For the LonBench-Cite evaluation, we prompt Llama-3.1-70B-Instruct to generate both the assistant response and corresponding citations. Then we evaluate the citations generated by Llama as well as the post-hoc citations generated by the LoRA adapter when invoked on the Llama responses. The results are shown in the table below:
+
+<table>
+<tr>
+    <th>Model used to generate response</th>
+    <th>Model used to generate citations</th>
+    <th colspan="3">Longbench-Chat (en)</th>
+    <th colspan="3">MultifieldQA (en)</th>
+    <th colspan="3">HotpotQA</th>
+    <th colspan="3">GovReport</th>
+</tr>
+<tr>
+    <th></th>
+    <th></th>
+    <th>R</th><th>P</th><th>F1</th>
+    <th>R</th><th>P</th><th>F1</th>
+    <th>R</th><th>P</th><th>F1</th>
+    <th>R</th><th>P</th><th>F1</th>
+</tr>
+<tr>
+    <td>Llama-3.1-70B-Instruct</td>
+    <td>Llama-3.1-70B-Instruct</td>
+    <td>27.0</td><td>34.4</td><td>26.1</td>
+    <td>46.1</td><td>63.3</td><td>49.7</td>
+    <td>34.0</td><td>39.4</td><td>30.2</td>
+    <td>55.0</td><td>77.5</td><td>62.0</td>
+</tr>
+<tr>
+    <td>Llama-3.1-70B-Instruct</td>
+    <td>Granite-3.3-8B LoRA citations</td>
+    <td>57.6</td><td>60.3</td><td>58.4</td>
+    <td>71.5</td><td>82.5</td><td>75.0</td>
+    <td>65.3</td><td>71.3</td><td>63.8</td>
+    <td>72.8</td><td>83.5</td><td>77.2</td>
+</tr>
+</table>
+
+We observe that the LoRA adapter performs across the board significantly better than Llama-3.1-70B-Instruct when prompted to create span-level citations. This demonstrates the value of the adapter to create post-hoc citations even for assistant responses generated by much bigger LLMs.
+
+Notes:
+- The evaluation results are reported on the English subset of LongBench-Cite (i.e., restricted to instances whose `language` field equals to `en`).
+- The results for the LoRA adapter do not include the performance for 4/585 tasks, which encountered out of memory errors. 
+- To prompt Llama to generate a response with citations, we use the one-shot prompt described in the paper.
+- For the LoRA adapter, sentence splitting of the context is performed using NLTK. For the response, we reuse the splitting in Llama's output (since the LongBench-Cite prompt instructs the model to output a response split into sentences/statements).
+
+## Model Card Authors 
+
+[Yannis Katsis](mailto:[email protected])</br>
+[Chulaka Gunasekara](mailto:[email protected])

citation_generation_lora/adapter_config.json

@@ -0,0 +1,34 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "ibm-granite/granite-3.3-8b-instruct",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_dropout": 0.1,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "q_proj",
+    "k_proj",
+    "v_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

citation_generation_lora/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3069aac1eccbaa522e921319a016a63a5d86370ce227f5bbe9912b7e74c45e2d
+size 99034536

citation_generation_lora/added_tokens.json

@@ -0,0 +1,9 @@
+{
+  "<|end_of_cite|>": 49156,
+  "<|end_of_plugin|>": 49158,
+  "<|end_of_role|>": 49153,
+  "<|start_of_cite|>": 49155,
+  "<|start_of_plugin|>": 49157,
+  "<|start_of_role|>": 49152,
+  "<|tool_call|>": 49154
+}

citation_generation_lora/special_tokens_map.json

@@ -0,0 +1,39 @@
+{
+  "additional_special_tokens": [
+    "<|start_of_role|>",
+    "<|end_of_role|>",
+    "<|tool_call|>",
+    "<|start_of_cite|>",
+    "<|end_of_cite|>",
+    "<|start_of_plugin|>",
+    "<|end_of_plugin|>"
+  ],
+  "bos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|end_of_text|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

citation_generation_lora/tokenizer_config.json

@@ -0,0 +1,235 @@
+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|end_of_text|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<fim_prefix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<fim_middle>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<fim_suffix>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<fim_pad>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<commit_before>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<commit_msg>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "17": {
+      "content": "<commit_after>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "18": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49152": {
+      "content": "<|start_of_role|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49153": {
+      "content": "<|end_of_role|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49154": {
+      "content": "<|tool_call|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49155": {
+      "content": "<|start_of_cite|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49156": {
+      "content": "<|end_of_cite|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49157": {
+      "content": "<|start_of_plugin|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "49158": {
+      "content": "<|end_of_plugin|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|start_of_role|>",
+    "<|end_of_role|>",
+    "<|tool_call|>",
+    "<|start_of_cite|>",
+    "<|end_of_cite|>",
+    "<|start_of_plugin|>",
+    "<|end_of_plugin|>"
+  ],
+  "bos_token": "<|end_of_text|>",
+  "chat_template": "{# Alias tools -> available_tools #}\n{%- if tools and not available_tools -%}\n    {%- set available_tools = tools -%}\n{%- endif -%}\n{%- if messages[0]['role'] == 'system' %}\n     {%- set system_message = messages[0]['content'] %}\n     {%- set loop_messages = messages[1:] %}\n {%- else %}\n     {%- set system_message = \"Knowledge Cutoff Date: April 2024.\nToday's Date: \" + strftime_now('%B %d, %Y') + \".\nYou are Granite, developed by IBM.\" %}\n     {%- if available_tools and documents %}\n         {%- set system_message = system_message + \" You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.\nWrite the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.\" %}\n     {%- elif available_tools %}\n         {%- set system_message = system_message + \" You are a helpful assistant with access to the following tools. When a tool is required to answer the user's query, respond only with <|tool_call|> followed by a JSON list of tools used. If a tool does not exist in the provided list of tools, notify the user that you do not have the ability to fulfill the request.\" %}\n     {%- elif documents %}\n         {%- set system_message = system_message + \" Write the response to the user's input by strictly aligning with the facts in the provided documents. If the information needed to answer the question is not available in the documents, inform the user that the question cannot be answered based on the available data.\" %}\n    {%- elif thinking %}\n    {%- set system_message = system_message + \" You are a helpful AI assistant.\nRespond to every user query in a comprehensive and detailed way. You can write down your thoughts and reasoning process before responding. In the thought process, engage in a comprehensive cycle of analysis, summarization, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process. In the response section, based on various attempts, explorations, and reflections from the thoughts section, systematically present the final solution that you deem correct. The response should summarize the thought process. Write your thoughts between <think></think> and write your response between <response></response> for each user query.\" %}\n     {%- else %}\n         {%- set system_message = system_message + \" You are a helpful AI assistant.\" %}\n     {%- endif %}\n     {%- if 'citations' in controls and documents %}\n         {%- set system_message = system_message + '\nUse the symbols <|start_of_cite|> and <|end_of_cite|> to indicate when a fact comes from a document in the search result, e.g <|start_of_cite|> {document_id: 1}my fact <|end_of_cite|> for a fact from document 1. Afterwards, list all the citations with their corresponding documents in an ordered list.' %}\n     {%- endif %}\n     {%- if 'hallucinations' in controls and documents %}\n         {%- set system_message = system_message + '\nFinally, after the response is written, include a numbered list of sentences from the response with a corresponding risk value that are hallucinated and not based in the documents.' %}\n     {%- endif %}\n     {%- set loop_messages = messages %}\n {%- endif %}\n {{- '<|start_of_role|>system<|end_of_role|>' + system_message + '<|end_of_text|>\n' }}\n {%- if available_tools %}\n     {{- '<|start_of_role|>available_tools<|end_of_role|>' }}\n     {{- available_tools | tojson(indent=4) }}\n     {{- '<|end_of_text|>\n' }}\n {%- endif %}\n {%- if documents %}\n     {%- for document in documents %}\n         {{- '<|start_of_role|>document {\"document_id\": \"' + document['doc_id'] | string + '\"}<|end_of_role|>\n' }}\n         {{- document['text'] }}\n         {{- '<|end_of_text|>\n' }}\n              {%- endfor %}\n {%- endif %}\n {%- for message in loop_messages %}\n     {{- '<|start_of_role|>' + message['role'] + '<|end_of_role|>' + message['content'] + '<|end_of_text|>\n' }}\n     {%- if loop.last and add_generation_prompt %}\n         {{- '<|start_of_role|>assistant' }}\n             {%- if controls %}\n                 {{- ' ' + controls | tojson()}}\n             {%- endif %}\n         {{- '<|end_of_role|>' }}\n     {%- endif %}\n {%- endfor %}",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|end_of_text|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 9223372036854775807,
+  "pad_token": "<|end_of_text|>",
+  "padding_side": "left",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|end_of_text|>",
+  "vocab_size": 49152
+}

citation_generation_lora/training_config.json

@@ -0,0 +1,93 @@
+{
+    "rl_model_name": null,
+    "rl_model_class": null,
+    "num_rollouts": 64,
+    "chunk_size": 16,
+    "ppo_epochs": 4,
+    "init_kl_coef": 0.05,
+    "num_layers_unfrozen": 2,
+    "rm_bits": 8,
+    "rl_lora_rank": 8,
+    "rl_lora_alpha": 32,
+    "rl_lora_dropout": 0.1,
+    "use_qlora_in_rl": false,
+    "use_rl_peft_config": true,
+    "lora_compute_dtype": "bfloat16",
+    "steps_per_print": 10,
+    "logdir": "aim-repo",
+    "aim_repo": null,
+    "experiment_name": "granite_3.3_prompted_citations_response_id_default_prompt",
+    "stage": 2,
+    "overlap_comm": false,
+    "contiguous_gradients": false,
+    "cpu_offload": false,
+    "optimizer": {
+        "optimizer_class": "FusedAdam",
+        "lr": 1e-05,
+        "weight_decay": 0.1,
+        "betas": [
+            0.9,
+            0.95
+        ],
+        "eps": 1e-10
+    },
+    "lr_schedule": "linear",
+    "warmup_steps": 200,
+    "datasets": [
+        {
+            "data_class": "JSONLinesDatasetRAGChat",
+            "data_name": "simulator-citations-chat",
+            "data_path": "data/citations/prompting_appraoch_3_3",
+            "data_sampling_proportion": 1,
+            "max_input_tokens": 5000,
+            "max_output_tokens": 512
+        }
+    ],
+    "seed": 42,
+    "training_inference_type": "lora_finetuning",
+    "prompt_tuning_init": null,
+    "prompt_tuning_init_text": null,
+    "num_virtual_tokens": null,
+    "load_path": null,
+    "peft_num_layers": null,
+    "peft_num_attention_heads": null,
+    "model_name": "ibm-granite/granite-3.3-8b-instruct",
+    "tokenizer_name": null,
+    "model_class": "AutoModelForCausalLM",
+    "gma_model_class": "Model",
+    "dtype": "bfloat16",
+    "trust_remote_code": false,
+    "padding_side": null,
+    "lora_rank": 16,
+    "lora_alpha": 16,
+    "lora_dropout": 0.1,
+    "lora_target_modules": [
+        "q_proj",
+        "k_proj",
+        "v_proj",
+        "o_proj",
+        "up_proj",
+        "down_proj",
+        "gate_proj"
+    ],
+    "save_huggingface_checkpoint": true,
+    "quantization_method": "fp4",
+    "bnb_4bit_use_double_quant": false,
+    "use_quantization_for_inference": false,
+    "max_seq_len": 2048,
+    "attention_implementation": "flash_attention_2",
+    "num_labels": 1,
+    "use_sdpa_attention": false,
+    "save_path": "checkpoints/granite_3.3_prompted_citations_default_prompt",
+    "ignore_sampling_proportion_for_validation": false,
+    "num_training_steps": 200000,
+    "gradient_accumulation_steps": 1,
+    "eval_interval": 20000,
+    "save_interval": 20000,
+    "batch_size_per_gpu": 1,
+    "coeff": 1.0,
+    "eval_during_training": true,
+    "smart_token_allocation": false,
+    "max_new_tokens": 0,
+    "gradient_checkpointing": true
+}