Luttrell S P, May 1996, Optimal posterior probabilities for self-organised neural networks, DRA technical report (Malvern, UK), DRA/CIS(SE1)/651/FUN/STIT/RP/1
This report presents some results for the explicit algebraic optimisation of encoder/decoder neural networks. The optimal form for the neuron response functions are derived in many cases. If the network is allowed to make only one attempt at encoding each input pattern then the optimal network is winner-take-all, whereas if more than one attempt is allowed then the optimal network has more than one winning neuron. These results on multiple attempts at encoding may be used to improve the performance of a vector quantiser without greatly increasing the size of the codebook, as would normally be the case. This is a probabilistic theory of vector quantisers.