README.md

---
license: apache-2.0
datasets:
- weizhiwang/mlm_filter_instructions
- Lin-Chen/ShareGPT4V
base_model:
- Qwen/Qwen2.5-1.5B-Instruct
- google/siglip-so400m-patch14-384
pipeline_tag: text-generation
language:
- zho
- eng
- fra
- spa
- por
- deu
- ita
- rus
- jpn
- kor
- vie
- tha
- ara
---


# MLM-Filter-Qwen2.5-1.5B-GPT4o Model Card

## Model details

**Model type:**
MLM-Filter-Qwen2.5-1.5B-GPT4o is an open-source MLLM trained to assess the data quality of image-text paired data. It can generate 4 quality metrics for image-text data: Image Text Matching, Object Detail Fulfillment, Caption Text Quality, and Semantic Understanding.

**Model date:**
MLM-Filter-Qwen2.5-1.5B-GPT4o was trained in Dec 2024.

**Paper or resources for more information:**
https://mlm-filter.github.io/

```
@article{wang2024finetuned,
  title={Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters},
  author={Wang, Weizhi and Mrini, Khalil and Yang, Linjie and Kumar, Sateesh and Tian, Yu and Yan, Xifeng and Wang, Heng},
  journal={arXiv preprint arXiv:2403.02677},
  year={2024}
}
```

## License
Apache-2.0

**Where to send questions or comments about the model:**
https://github.com/Victorwz/MLM_Filter/issues

## Intended use
**Primary intended uses:**
MLM-Filter can be used as a drop-in replacement for CLIPScore in these tasks:

1. Score image-text data in large-scale pre-training dataset and then filter high-quality subsets based on the scores (For training MLLMs or VLMs, please consider to jointly use the Image-Text Matching score and the Object Detail Fulfillment score);

2. Evaluate the image-text alignment for image2text or text2image generation models;

3. Any potential applications with the need to calculate the image-text alignment.


## Training dataset (709K)
- 665k ShareGPT4V data.
- 44k instructions on image-text data quality assessment tasks ranging across 4 metrics.

## Usage Sample
Please follow the instructions in https://github.com/Victorwz/MLM_Filter.