Not working with latest latest trunk version of NeMo?
I had the huggingface version (frame_vad_multilingual_marblenet_v2.0.nemo) working beautifully and predictably with pip3 install nemo_toolkit[asr], but then I needed to upgrade to pip3 install "nemo_toolkit[asr,tts] @ git+https://github.com/NVIDIA/NeMo.git" to use the Canary-Qwen model, now I get the error RuntimeError: Error(s) in loading state_dict for EncDecClassificationModel: Unexpected key(s) in state_dict: "loss.weight".
I tried downloading the latest model from NGC (which shows as 1.20.0?) using wget --content-disposition 'https://api.ngc.nvidia.com/v2/models/org/nvidia/team/nemo/vad_multilingual_frame_marblenet/1.20.0/files?redirect=true&path=vad_multilingual_frame_marblenet.nemo' -o 'vad_multilingual_frame_marblenet.nemo' but get the same error
Oddly self.model = nemo_asr.models.EncDecClassificationModel.from_pretrained(model_name="vad_multilingual_marblenet") seems to load without an error, however I get this warning "Please use the EncDecSpeakerLabelModel instead of this model. EncDecClassificationModel model is kept for backward compatibility with older models." And then when the model runs it is completely unpredictable sometimes taking half a second to register silence while othertimes it takes six seconds. I didn't have this issue at first.
Is there a certain model I should be using for compatibility with that latest nemo_toolkit[asr] directly from the github?
- The
vad_multilingual_marblenet modelis an older version. The updatedframe_vad_multilingual_marblenet_v2.0.nemomodel is only available on Hugging Face. - As a quick workaround, set
strict=Falsewhen loading the model to skip the loss.weight parameter.
Update to main as well, there is a recent fix merged to main related to frame vad models
Thank you for the clarification. setting strict=False worked. Was already on main, tried updating again but problem persists without the strict=False flag.