how to exclude file in "original" folder while model download

#122
by meetzuber - opened

Hi

when I am tring to run gpt-oss model using vllm, it downloads model from hugging face. it also download original folder which takes too much time and also consume additional disk space. original folder is not used while inferencing the model.

Please suggest work around for this.

Thanks

You could use the huggingface_hub.snapshot_download() function to pre-download the model, which allows you to filter out glob patterns for directories/files: https://huggingface.co/docs/huggingface_hub/en/guides/download#filter-files-to-download

Then when you run vLLM you can specify HF_HOME or HF_HUB_CACHE to point to the model path: https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables#hfhome

Sign up or log in to comment