Create README.md

README.md

@@ -0,0 +1,41 @@
+---
+license: other
+license_name: inf
+license_link: https://huggingface.co/infly/OpenCoder-1.5B-Base/blob/main/LICENSE
+language:
+- en
+- zh
+base_model: infly/OpenCoder-1.5B-Base
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- code
+---
+
+<h1 align="center">
+  <br>
+    OpenCoder-1.5B-Base-32K-via-16K
+  <br>
+</h1>
+
+<p align="center">
+  <a href="https://github.com/sapromak/adaptive-code-completion">Home Page</a> •
+  <a href="https://huggingface.co/collections/sapromak/repository-level-pre-trained-opencoder-684206bfc99d48a7e94c0789">Collection</a> •
+  <a href="https://openreview.net/forum?id=t9RN9WX4Ic">Paper</a> •
+  <a href="https://github.com/sapromak/adaptive-code-completion/blob/main/thesis.pdf">Thesis</a>
+</p>
+
+## Description
+
+This model is derived from [OpenCoder-1.5B-Base](https://huggingface.co/infly/OpenCoder-1.5B-Base) by applying an additional context extension fine-tuning with an Adjustment of the Base Frequency parameter of RoPE from 10,000 to 500,000. The number of optimization steps is 512 with a batch size of 128 on sequences of 16,384 length. The repository context is composed based on the _Path Distance_ heuristics, more details on which and other aspects including all code used can be found on the [Home Page](https://github.com/sapromak/adaptive-code-completion) of the project. Note that this model is created with the intent to answer specific research questions and __not__ to gain the maximum possible performance on the repository-level code completion setup. Consider it more as a baseline.
+
+<div align="center">
+  <img src="https://github.com/sapromak/adaptive-code-completion/blob/main/paper/figures/compilation/beyond-training-window/beyond-training-window-inproject.svg?raw=true" width="70%" alt="Performance" />
+  <p>Exact Match on the <em>inproject</em> lines of the <em>large-context</em> subset of the <a href="https://huggingface.co/datasets/JetBrains-Research/lca-project-level-code-completion">Project-Level Code Completion task</a> from the <a href="https://arxiv.org/abs/2406.11612">Long Code Arena benchmark</a>. This checkpoint (solid orange curve) demonstrates its best performance at a context length of 32,768. "1K" refers to 1,024 tokens. The star markers denote the context length used during the repository-level pre-training stage.
+</div>
+
+## Quickstart
+
+```python
+# TODO
+```