QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichment

This repository hosts the official model weights and evaluation benchmarks for QueenVIS.

🔗 Quick Links

• Project Page: aut-aisl.github.io/QueenVIS-Project-Page
• GitHub Repository: github.com/ArianKheir/QueenVIS
• Paper (arXiv): 10.48550/arXiv.2607.24598

---

📖 Model Description

QueenVIS rethinks the paradigm of image-only training for Video Instance Segmentation (VIS). By introducing training-only auxiliary heads for feature prediction and center prediction, QueenVIS embeds dense appearance and spatial priors directly into transformer object queries without requiring video clip supervision.

• Key Feature: Adds zero parameters and zero extra GFLOPs at inference relative to image-only baselines like MinVIS.
• Temporal Association: Employs a training-free confidence-guided query propagation mechanism paired with a non-parametric memory bank to maintain temporal identity across frames.

---

📊 Datasets & Evaluation Metrics

QueenVIS is evaluated across standard VIS benchmarks:

• YouTube-VIS (2019 / 2021 / 2022 Long Split): Standard and long-sequence video benchmarks.
• OVIS (Occluded Video Instance Segmentation): A challenging benchmark characterized by severe occlusion and complex object interactions.

Performance is measured using standard Video Instance Segmentation metrics:

• AP (Average Precision): Primary metric across IoU thresholds.
• AP₅₀ & AP₇₅: Average Precision at IoU thresholds of 0.50 and 0.75.
• AR₁ & AR₁₀: Average Recall given 1 and 10 detections per video.

---

📈 Evaluation Results

1. Main Benchmarks (ResNet-50 Backbone)

Dataset	AP	AP₅₀	AP₇₅	AR₁	AR₁₀
YouTube-VIS 2019	51.8	75.2	58.3	49.4	62.1
YouTube-VIS 2021	50.9	72.1	55.8	42.8	57.3
YouTube-VIS 2022 (Long)	33.6	56.6	34.0	31.7	39.9
OVIS (Occluded VIS)	29.8	52.7	29.0	15.4	34.6

---

2. Scaling to Vision Transformers (Swin-L Backbone)

Dataset	AP	AP₅₀	AP₇₅	AR₁	AR₁₀
YouTube-VIS 2019	63.2	85.3	68.3	55.5	68.4
YouTube-VIS 2021	59.8	81.5	65.2	48.3	64.2
OVIS (Occluded VIS)	41.0	62.2	43.0	18.4	45.0

---

🚀 How to Use

For complete instructions on installation, environment setup, dataset preparation, and inference scripts, visit the official GitHub Repository.

---

📚 Citation

If you find QueenVIS useful in your research, please cite our work:

@misc{kheirandish2026queenvis,
      title={QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichment}, 
      author={Arian Kheirandish and Fardin Ayar and Ehsan Javanmardi and Manabu Tsukada and Mahdi Javanmardi},
      year={2026},
      eprint={2607.24598},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={[https://doi.org/10.48550/arXiv.2607.24598](https://doi.org/10.48550/arXiv.2607.24598)}
}