Update summary of the tasks (#16528)
Update summary of the tasks (#16528)

* 📝 add image/vision classification and asr

* 🖍 minor formatting fixes

* Fixed a typo in legacy seq2seq_trainer.py (#16531)

* Add ONNX export for BeiT (#16498)

* Add beit onnx conversion support

* Updated docs

* Added cross reference to ViT ONNX config

* call on_train_end when trial is pruned (#16536)

* Type hints added (#16529)

* Fix Bart type hints (#16297)

* Add type hints to PLBart PyTorch

* Remove pending merge conflicts

* Fix PLBart Type Hints

* Add changes from review

* Add VisualBert type hints (#16544)

* Adding missing type hints for mBART model (PyTorch) (#16429)

* added type hints for mbart tensorflow tf implementation

* Adding missing type hints for mBART model 

Tensorflow Implementation model added with missing type hints

* Missing Type hints - correction

For TF model

* Code fixup using make quality tests

* Hint types - typo error

* make fix-copies and make fixup

* type hints

* updated files

* type hints update

* making dependent modesls coherent

Co-authored-by: matt <151604bfbded75a2c68398e74f7598899a05c8f1@gmail.com>

* Remove MBart subclass of XLMRoberta in tokenzier docs (#16546)

* Remove MBart subclass of XLMRoberta in tokenzier

* Fix style

* Copy docs from MBart50 tokenizer

* Use random_attention_mask for TF tests (#16517)

* use random_attention_mask for TF tests

* Fix for TFCLIP test (for now).

Co-authored-by: ydshieh <b8b5afdcc4de58b54ce28738b50f86ec989da691@users.noreply.github.com>

* Improve code example (#16450)

Co-authored-by: Niels Rogge <01b44cd62738107b4ea973555227cc43b36c63bb@nielss-mbp.home>

* Pin tokenizers version <0.13 (#16539)

* Pin tokenizers version <0.13

* Style

* Add code samples for TF speech models (#16494)

Co-authored-by: ydshieh <b8b5afdcc4de58b54ce28738b50f86ec989da691@users.noreply.github.com>

* [FlaxSpeechEncoderDecoder] Fix dtype bug (#16581)

* [FlaxSpeechEncoderDecoder] Fix dtype bug

* more fixes

* Making the impossible to connect error actually report the right URL. (#16446)

* Fix flax import in __init__.py: modeling_xglm -> modeling_flax_xglm (#16556)

* Add utility to find model labels (#16526)

* Add utility to find model labels

* Use it in the Trainer

* Update src/transformers/utils/generic.py

Co-authored-by: Matt <4fcd5525d578c321218f87bf53634045b53f90e6@users.noreply.github.com>

* Quality

Co-authored-by: Matt <4fcd5525d578c321218f87bf53634045b53f90e6@users.noreply.github.com>

* Enable doc in Spanish (#16518)

* Reorganize doc for multilingual support

* Fix style

* Style

* Toc trees

* Adapt templates

* Add use_auth to load_datasets for private datasets to PT and TF examples (#16521)

* fix formatting and remove use_auth

* Add use_auth_token to Flax examples

* add a test checking the format of `convert_tokens_to_string`'s output (#16540)

* add new tests

* add comment to overridden tests

* TF: Finalize `unpack_inputs`-related changes (#16499)

* Add unpack_inputs to remaining models

* removed kwargs to `call()` in TF models

* fix TF T5 tests

* [SpeechEncoderDecoderModel] Correct Encoder Last Hidden State Output (#16586)

* initialize the default rank set on TrainerState (#16530)

* initialize the default rank set on TrainerState

* fix style

* Trigger doc build

* Fix CI: test_inference_for_pretraining in ViTMAEModelTest (#16591)

Co-authored-by: ydshieh <b8b5afdcc4de58b54ce28738b50f86ec989da691@users.noreply.github.com>

* add a template to add missing tokenization test (#16553)

* add a template to add missing tokenization test

* add cookiecutter setting

* improve doc

* Update templates/adding_a_missing_tokenization_test/README.md

Co-authored-by: Sylvain Gugger <222c4d44cc8ec39802c192ed0aa3f97cdb77ff4e@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <222c4d44cc8ec39802c192ed0aa3f97cdb77ff4e@users.noreply.github.com>

* made _load_pretrained_model_low_mem static + bug fix (#16548)

* handle torch_dtype in low cpu mem usage (#16580)

* [Doctests] Correct filenaming (#16599)

* [Doctests] Correct filenaming

* improve quicktour

* make style

* Adding new train_step logic to make things less confusing for users (#15994)

* Adding new train_step logic to make things less confusing for users

* DO NOT ASK WHY WE NEED THAT SUBCLASS

* Metrics now working, at least for single-output models with type annotations!

* Updates and TODOs for the new train_step

* Make fixup

* Temporary test workaround until T5 has types

* Temporary test workaround until T5 has types

* I think this actually works! Needs a lot of tests though

* MAke style/quality

* Revert changes to T5 tests

* Deleting the aforementioned unmentionable subclass

* Deleting the aforementioned unmentionable subclass

* Adding a Keras API test

* Style fixes

* Removing unneeded TODO and comments

* Update test_step too

* Stop trying to compute metrics with the dummy_loss, patch up test

* Make style

* make fixup

* Docstring cleanup

* make fixup

* make fixup

* Stop expanding 1D input tensors when using dummy loss

* Adjust T5 test given the new compile()

* make fixup

* Skipping test for convnext

* Removing old T5-specific Keras test now that we have a common one

* make fixup

* make fixup

* Only skip convnext test on CPU

* Update src/transformers/modeling_tf_utils.py

Co-authored-by: Sylvain Gugger <222c4d44cc8ec39802c192ed0aa3f97cdb77ff4e@users.noreply.github.com>

* Update src/transformers/modeling_tf_utils.py

Co-authored-by: Sylvain Gugger <222c4d44cc8ec39802c192ed0aa3f97cdb77ff4e@users.noreply.github.com>

* Avoiding TF import issues

* make fixup

* Update compile() to support TF 2.3

* Skipping model.fit() on template classes for now

* Skipping model.fit() on template class tests for now

* Replace ad-hoc solution with find_labels

* make fixup

Co-authored-by: Sylvain Gugger <222c4d44cc8ec39802c192ed0aa3f97cdb77ff4e@users.noreply.github.com>

* Adding missing type hints for BigBird model   (#16555)

* added type hints for mbart tensorflow tf implementation

* Adding missing type hints for mBART model 

Tensorflow Implementation model added with missing type hints

* Missing Type hints - correction

For TF model

* Code fixup using make quality tests

* Hint types - typo error

* make fix-copies and make fixup

* type hints

* updated files

* type hints update

* making dependent modesls coherent

* Type hints for BigBird

* removing typos

Co-authored-by: matt <151604bfbded75a2c68398e74f7598899a05c8f1@gmail.com>

* [deepspeed] fix typo, adjust config name (#16597)

* 🖍 apply feedback

Co-authored-by: Cathy <c6e415a3c31aa3f6b8ca0ae81b2f761f80f8a6ad@qq.com>
Co-authored-by: Jim Rohrer <93b3ef216e7480f10625472781b89844c1af184f@gmail.com>
Co-authored-by: Ferdinand Schlatt <a0b729c854c9dbf2424b441389dbbcb516bddad8@gmail.com>
Co-authored-by: Dahlbomii <7f0d4d5b0b300e71ca4db2647b70246ae4363d37@users.noreply.github.com>
Co-authored-by: Gunjan Chhablani <1efb5c6a85bb7aea0786ec2484494cefd57ef3a1@gmail.com>
Co-authored-by: Rishav Chandra Varma <50921dfe9cbcaf1a50bdcfe44e556ceb222c4a1c@iiits.in>
Co-authored-by: matt <151604bfbded75a2c68398e74f7598899a05c8f1@gmail.com>
Co-authored-by: Yih-Dar <2521628+b8b5afdcc4de58b54ce28738b50f86ec989da691@users.noreply.github.com>
Co-authored-by: ydshieh <b8b5afdcc4de58b54ce28738b50f86ec989da691@users.noreply.github.com>
Co-authored-by: NielsRogge <e45a0ab1d7a9729bf40010a5d71678f7e86d039d@users.noreply.github.com>
Co-authored-by: Niels Rogge <01b44cd62738107b4ea973555227cc43b36c63bb@nielss-mbp.home>
Co-authored-by: Lysandre Debut <1974fe798614508b0cab7f3deceace64fb8dcf09@reseau.eseo.fr>
Co-authored-by: Patrick von Platen <3f65d0f67d3758130584b6bd9911c60cc3d4ded4@gmail.com>
Co-authored-by: Nicolas Patry <a2be6f5dcd3a80f15653da487ec84559830cbf7a@protonmail.com>
Co-authored-by: Daniel Stancl <0867f6bd04111efdad5381280abee7e2a3b1a987@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <222c4d44cc8ec39802c192ed0aa3f97cdb77ff4e@users.noreply.github.com>
Co-authored-by: Matt <4fcd5525d578c321218f87bf53634045b53f90e6@users.noreply.github.com>
Co-authored-by: Karim Foda <073cbd6faa2d4b7b19a1543f5ebec3e11a632f17@users.noreply.github.com>
Co-authored-by: SaulLu <8baabefdbad237c29b328ce329542df23eca2638@users.noreply.github.com>
Co-authored-by: Joao Gante <c510cd8607f92e1e09fd0b0d0d035c16d2428fa4@huggingface.co>
Co-authored-by: Sanchit Gandhi <85685637b43f120671004d3398d8280d5bdf147a@users.noreply.github.com>
Co-authored-by: Andres Codas <118d67225f395d10b45d23eec9a5abce6662c476@users.noreply.github.com>
Co-authored-by: Sylvain Gugger <e007b306d45b92d0641fbc7b2cc384bdafe92fd8@gmail.com>
Co-authored-by: Francesco Saverio Zuppichini <3e0e2ebcdfcd75a4625593f6b7dfb0e027c10043@gmail.com>
Co-authored-by: Suraj Patil <e55124f2bc241ddfef100229e12a1887b326eb73@gmail.com>
Co-authored-by: Stas Bekman <7755b8071ff7368c870e3a79649c5b25a8766143@users.noreply.github.com>
huggingface/transformers,huggingface/transformers,huggingface/transformers,huggingface/transformers