Does fixed hyperparameters perform well regardless the number of training examples? I'm new in this community and I don't know whether my question is proper for this community. I will delete this post if it is not proper.

I'm interested in deep learning network models and have a question about this.

Suppose we have a feedforward neural network models with different hyperparameters (for example, the number of layers, neurons and so on), say $H_1,..., H_n$. Also assume all the samples are selected from a particular distribution.

First we got $100$ samples. If we find that a typical hyperparameter set, $H_i$, performs best. Now I want to increase the number of samples to $1000, 10000,$ even millions. Is $H_i$ still the best hyperparameter set to perform among $H_i$'s? I want to know whether there are references about such topics. The best is a mathematical proof about this fact.

Again, please tell me if this question is not proper to the community. Thanks a lot!
No. You're asking about model selection. Larger sample sizes will allow you to choose more complex models. Read up on the keywords overfit/underfit, model selection, Structural Risk Minimization, and in particular this article:
https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=705570