Before saving the model, the `save_directory` is cleaned from any previous shard files.

<Tip warning={true}>

If one of the model's tensor is bigger than `max_shard_size`, it will end up in its own shard which will have a
size greater than `max_shard_size`.

</Tip>

<Tip warning={true}>