Can you help me to give code of finetuning the embedding model according to our own dataset

#20
by RohitUltimate - opened

File "d:\Rohit\Accabot\trainingST.py", line 98, in
model.fit(
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\sentence_transformers\fit_mixin.py", line 408, in fit
trainer.train(resume_from_checkpoint=resume_from_checkpoint)
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\transformers\trainer.py", line 1424, in train
return inner_training_loop(
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\transformers\trainer.py", line 1506, in _inner_training_loop
self._run_epoch(
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\transformers\trainer.py", line 1734, in run_epoch
tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\transformers\trainer.py", line 1934, in training_step
self.accelerator.backward(loss, **kwargs)
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\accelerate\accelerator.py", line 2834, in backward
self.scaler.scale(loss).backward(**kwargs)
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\torch_tensor.py", line 647, in backward
torch.autograd.backward(
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\torch\autograd_init
.py", line 354, in backward
_engine_run_backward(
File "C:\Users\CT_USER\anaconda3\envs\qwen_3.5_unsloth\lib\site-packages\torch\autograd\graph.py", line 829, in _engine_run_backward
return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: element 0 of tensors does not require grad and does not have a grad_fn

Jina AI org

Hey, this generally happens when there are no trainable parameters in the model. So you should set the train mode.

I would suggest fine-tuning single-task versions of the model, which have more standard model implementations, and should be easier to integrate with ST Trainer:
https://huggingface.co/jinaai/jina-embeddings-v5-text-small-retrieval
https://huggingface.co/jinaai/jina-embeddings-v5-text-small-classification
...

If it still doesn't work, feel free to share your code and I'll see what is the issue

Sign up or log in to comment

Free AI Image Generator No sign-up. Instant results. Open Now