Sample complexity for learning Boltzmann Distribution parameters I am trying to think through the number of samples that I would need to estimate the parameters of a Boltzmann partition function to a desirable precision.
Suppose that there are $N$ possible states of the world, with the probability of state $i$ being observed equal to
$$Pr(i | \theta) = \frac{e^{-\theta_i}}{ \sum_{i=1}^N e^{-\theta_i}}.$$
I don't know what the values of $\{\theta_i\}_{i=1}^N$ are, but I can sample independent observations from the set $\{1,\dots,N\}$ of possible states of the world, distributed according to the distribution $Pr(i)$ given above.
Let's say I draw $m$ observations $X_1,...,X_m \in \{1,...,N\}$, and define  the maximum likelihood estimator
$$\widehat{\theta} = \arg \max_{\theta'}  \sum_{j=1}^m  \log Pr(X_j | \theta)$$
When $m \to \infty$, then  $\widehat{\theta} = \theta$. However, I'm not sure how many samples $m$ I would need to ensure that the estimate $\widehat{\theta}$ is probably approximately correct. That is, how large does $m$ need to be in terms of $\epsilon,N,\delta$ to ensure that   $Pr ( \| \widehat{\theta} - \theta \|_{\infty} &gt; \epsilon ) &lt; \delta$?
Under appropriate smoothness conditions (satisfied by the Boltzmann distribution), the maximum likelihood is asymptotically normal. Meaning: the vector $\sqrt n(\theta-\hat\theta)$ will have approximately normal distribution, with mean $0$ and covariance matrix given by the inverse Fisher matrix, see Section 9 here:
http://www.stat.cmu.edu/~larry/=stat705/Lecture9.pdf