Can't Run The Demo FT Code? "Cannot copy out of meta tensor"

#42
by alentzy - opened

Hello,

I'm following this cookbook entry to the the letter: https://cookbook.openai.com/articles/gpt-oss/fine-tune-transfomers and no matter what I do I end up with the same error when creating the SFTTrainer:

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[7], line 4
      1 from trl import SFTTrainer
      2 peft_model._hf_peft_config_loaded = True
----> 4 trainer = SFTTrainer(
      5     model=peft_model,
      6     args=training_args,
      7     train_dataset=dataset,
      8     processing_class=tokenizer,
      9 )

NotImplementedError: Cannot copy out of meta tensor; no data! Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.

I'm on an RTX A6000 GPU. I've also run the same cookbook code in the same environment with Llama 3.1 8B without the error. Anyone else have this issue or know of a fix?

Sign up or log in to comment