Luttrell S P, October 1997, A unified theory of density models and auto-encoders, DERA technical report (Malvern, UK), DERA/CIS/CIS5/TR97303
This report introduces an objective function for simultaneously optimising the density model and transition matrices of a Markov source. The chosen objective function seeks to minimise the average total number of bits that is required to encode the joint state of the Markov source. This may be applied to the problem of optimising the bottom-up (recognition model) and top-down (generative model) connections in a multilayer neural network. This approach unifies many previous results on the optimisation of multilayer unsupervised neural networks.