File size: 1,747 Bytes
e90c14c |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"machine_shape": "hm",
"gpuType": "T4",
"provenance": []
},
"accelerator": "GPU",
"kaggle": {
"accelerator": "gpu"
},
"language_info": {
"name": "python"
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "code",
"source": [
"!pip install -U transformers"
],
"metadata": {
"id": "4LXugJ1wUN5o"
},
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Local Inference on GPU\n",
"Model page: https://huggingface.co/vikhyatk/moondream2\n",
"\n",
"⚠️ If the generated code snippets do not work, please open an issue on either the [model repo](https://huggingface.co/vikhyatk/moondream2)\n",
"\t\t\tand/or on [huggingface.js](https://github.com/huggingface/huggingface.js/blob/main/packages/tasks/src/model-libraries-snippets.ts) 🙏"
],
"metadata": {
"id": "t490XrsVUN5q"
}
},
{
"cell_type": "code",
"source": [
"from transformers import AutoModelForCausalLM, AutoTokenizer\n",
"\n",
"tokenizer = AutoTokenizer.from_pretrained(\"vikhyatk/moondream2\")\n",
"model = AutoModelForCausalLM.from_pretrained(\n",
" \"vikhyatk/moondream2\",\n",
" revision=\"2025-06-21\",\n",
" trust_remote_code=True,\n",
" device_map={\"\": \"cuda\"}\n",
")"
],
"metadata": {
"id": "6aWNuxB8UN5s"
},
"execution_count": null,
"outputs": []
}
]
} |