EM for exponential families

Suppose the complete density is of the form:

$$g_c(x;\psi)=b(x)e^{(\psi)'t(x)}/a(\psi)$$

We will suppose the model of full rank, ie the sufficient statistic $t(x)$ to be of the same dimension as $\psi$.

$$E_\psi[t(X)]=\frac{\partial log a(\psi)}{\partial \psi}$$

$$Q(\psi,\psi^{(k)})=\psi'E_{\psi^{(k)}}[t(X)\vert y]-log a(\psi)$$

Differentiate with regards to $\psi$ and you find that to maximise $Q$ you must take:

$$E_\psi[t(X)]=t^{(k)}=E_{\psi^{(k)}}[t(X)\vert y]$$

so we need to solve:

$$\partial{log a(\psi)}{\partial \psi}=E_{\psi^{(k)}}[t(X)\vert y]$$

As an application, you can consider any multinomial as an exponential family, (see Lehmann, Theory of Point Estimation, page 28), so that they all enter this case.

A particular example was the one I did the first day where the new parameter space had a natural interpretation, that of the gene counting example, where the first mutinomial is the observed one of the frequencies of the observed phenotypes. $y=(n_A,n_B,n_{AB},n_O)$, the parameter of interest is the probability of each of the genotypes $A,B,O=(p,q,r)$, we can usethe augmented model: $x=(n_{AA},n_{BA},n_{BB},n_{OO})$

Here is a link to some very serious papers about tomography: http://www.stats.bris.ac.uk/pub/reports/TOMO.

And here is a link to a nice application of the EM to Bayesian self-organizing map simulations: http://www.aist.go.jp/NIBH/~b0616/Lab/BSOM1/