(************** Content-type: application/mathematica ************** CreatedBy='Mathematica 5.0' Mathematica-Compatible Notebook This notebook can be used with any Mathematica-compatible application, such as Mathematica, MathReader or Publicon. The data for the notebook starts with the line containing stars above. To get the notebook into a Mathematica-compatible application, do one of the following: * Save the data starting with the line of stars above into a file with a name ending in .nb, then open the file inside the application; * Copy the data starting with the line of stars above to the clipboard, then use the Paste menu command inside the application. Data for notebooks contains only printable 7-bit ASCII and can be sent directly in email or through ftp in text mode. Newlines can be CR, LF or CRLF (Unix, Macintosh or MS-DOS style). NOTE: If you modify the data for this notebook not in a Mathematica- compatible application, you must delete the line below containing the word CacheID, otherwise Mathematica-compatible applications may try to use invalid cache data. For more information on notebooks and Mathematica-compatible applications, contact Wolfram Research: web: http://www.wolfram.com email: info@wolfram.com phone: +1-217-398-0700 (U.S.) Notebook reader applications are available free of charge from Wolfram Research. *******************************************************************) (*CacheID: 232*) (*NotebookFileLineBreakTest NotebookFileLineBreakTest*) (*NotebookOptionsPosition[ 93976, 3007]*) (*NotebookOutlinePosition[ 104178, 3272]*) (* CellTagsIndexPosition[ 102042, 3213]*) (*WindowFrame->Normal*) Notebook[{ Cell[CellGroupData[{ Cell["Notes", "Section 1"], Cell[CellGroupData[{ Cell["Editorial Changes", "Subsection"], Cell["American spelling changed to British spelling.", "Text"], Cell["\[Section] and \[Section]\[Section] changed to section.", "Text"], Cell["\<\ Equation numbers in appendix continue on from equation numbers in the body of \ the paper.\ \>", "Text"], Cell[TextData[{ Cell[BoxData[ FormBox[ ButtonBox["TYPO", ButtonData:>"Ed:Change6", Active->True, ButtonStyle->"Hyperlink"], TextForm]]], "Equation reformatted to make it easier to read." }], "Text"], Cell[TextData[{ Cell[BoxData[ FormBox[ ButtonBox["TYPO", ButtonData:>"Eq:ScatteringPDF", Active->True, ButtonStyle->"Hyperlink"], TextForm]]], "\"", Cell[BoxData[ \(TraditionalForm\`exp(\(-f\^\[Dagger]\) \(\[Sigma]\^\(-1\)\) f]\)]], "\" changed to \"", Cell[BoxData[ \(TraditionalForm\`exp(\(-f\^\[Dagger]\) \(\[Sigma]\^\(-1\)\) f)\)]], "\"." }], "Text"], Cell[TextData[{ Cell[BoxData[ FormBox[ ButtonBox["TYPO", ButtonData:>"Ed:Change2", Active->True, ButtonStyle->"Hyperlink"], TextForm]]], "\"vectors\" changed to \"", StyleBox["vectors", FontSlant->"Italic"], "\"", StyleBox[".", FontSlant->"Italic"] }], "Text"], Cell[TextData[{ Cell[BoxData[ FormBox[ ButtonBox["TYPO", ButtonData:>"Ed:Change3", Active->True, ButtonStyle->"Hyperlink"], TextForm]]], Cell[BoxData[ \(TraditionalForm\`\[Sum]\&m\)]], "\" changed to \"", Cell[BoxData[ \(TraditionalForm\`\[Sum]\+m\)]], "\"." }], "Text"], Cell[TextData[{ Cell[BoxData[ FormBox[ ButtonBox["TYPO", ButtonData:>"Ed:Change4", Active->True, ButtonStyle->"Hyperlink"], TextForm]]], "\"727\" changed to \"727-746\"." }], "Text"], Cell[TextData[{ Cell[BoxData[ FormBox[ ButtonBox["TYPO", ButtonData:>"Ed:Change5", Active->True, ButtonStyle->"Hyperlink"], TextForm]]], "\"(Oxford: Oxford University Press).\" changed to \"(Oxford: Oxford \ University Press), pp. 739-747.\"." }], "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell["\<\ The theory of Bayesian super-resolution of coherent images: a review\ \>", "Title"], Cell["\<\ S. P. LUTTRELL Royal Signals and Radar Establishment, St Andrews Road, Malvern Worcestershire WR14 3PS, England\ \>", "Author"], Cell["\<\ This paper appeared in International Journal of Remote Sensing, 1991, vol. \ 12, no. 2, pp. 303-314.\ \>", "Text"], Cell["\[Copyright] Controller, Her Majesty's Stationery Office, 1991 ", "Text"], Cell[TextData[{ StyleBox["Abstract.", FontWeight->"Bold"], " We review some theoretical work on super-resolution of coherent images \ from a Bayesian point of view. The well known singular value decomposition \ super-resolution method emerges as a special case, and it is extended in \ order to derive a practical iterative super-resolution algorithm." }], "Abstract"], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", " Introduction" }], "Section 1"], Cell["\<\ Super-resolution is the name given to techniques which enhance the resolving \ power of imaging systems beyond that permitted by the Rayleigh resolution \ criterion. This is achieved at the cost of having to introduce prior \ knowledge about the object which is being imaged.\ \>", "Text"], Cell[TextData[{ "In order to take account of prior knowledge in a consistent fashion we \ shall derive the super-resolution equations from a Bayesian point of view. \ Denote the scattering and imaging operations as ", Cell[BoxData[ \(TraditionalForm\`\[Sigma] \[RightArrow] \(f \[RightArrow] g\)\)]], " where the cross-section ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " produces a scattered field ", Cell[BoxData[ \(TraditionalForm\`f\)]], ", which is then imaged to produce an image ", Cell[BoxData[ \(TraditionalForm\`g\)]], ". Bayes' theorem may then be used to solve various inverse problems posed \ in terms of these quantities. We shall study two such problems." }], "Text"], Cell[TextData[{ "The first is the inverse scattered field problem (recovering ", Cell[BoxData[ \(TraditionalForm\`f\)]], " from ", Cell[BoxData[ \(TraditionalForm\`g\)]], ") which becomes ", Cell[BoxData[ \(TraditionalForm\`Pr( f | g) \[Proportional] \(Pr(g | f)\) \(Pr(f)\)\)]], " where ", Cell[BoxData[ \(TraditionalForm\`Pr(f)\)]], " represents our prior knowledge of the scattered field (which depends on \ our prior estimate of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], "). In particular, when the probability density functions (p.d.f.s) are \ Gaussian, the peak of ", Cell[BoxData[ \(TraditionalForm\`Pr(f | g)\)]], " gives the minimum mean square error (MMSE in the Euclidean sense) \ estimate of ", Cell[BoxData[ \(TraditionalForm\`f\)]], " given ", Cell[BoxData[ \(TraditionalForm\`g\)]], ". This is the basis of a simple and successful super-resolution \ technique." }], "Text"], Cell[TextData[{ "The second is the inverse cross-section problem (recovering ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " from ", Cell[BoxData[ \(TraditionalForm\`g\)]], ") which becomes ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma] | g) \[Proportional] \(Pr( g | \[Sigma])\) \(Pr(\[Sigma])\)\)]], ". The peak of ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma] | g)\)]], " can be found by a hill-climbing approach in which inverse problem 1 is \ solved repeatedly for various estimates of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], "." }], "Text"] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", " Fundamentals" }], "Section"], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " A resum\[EAcute] of Bayesian inference" }], "Subsection"], Cell[TextData[{ "Throughout this paper we shall adopt a Bayesian approach to \ super-resolution. The motivation for this is consistency of inference, which \ is afforded uniquely by Bayesian calculus. The notions of Bayesian calculus \ are highly non-trivial, so we shall first of all review them. In particular, \ we shall explain what a Bayesian probability is, and how it is (usually) \ different from a frequentist probability (", ButtonBox["Jeffries 1939", ButtonData:>"Ref:Jeffreys1939", ButtonStyle->"Hyperlink"], ")." }], "Text"], Cell[TextData[{ "We shall now summarise a simple argument in favour of the Bayesian \ approach which is due to ", ButtonBox["Skilling (1989)", ButtonData:>"Ref:Skilling1989", ButtonStyle->"Hyperlink"], ", which is, in turn, based on ", ButtonBox["Cox (1946)", ButtonData:>"Ref:Cox1946", ButtonStyle->"Hyperlink"], ". This summary is intended to be merely an indication of the simplicity \ and elegance of the axioms from which all of Bayesian calculus flows. Thus, \ impose the following consistency conditions on the probabilities related to a \ pair of states ", Cell[BoxData[ \(TraditionalForm\`f\)]], " and ", Cell[BoxData[ \(TraditionalForm\`g\)]], " of an arbitrary system" }], "Text"], Cell[BoxData[{ \(TraditionalForm\`prefer\ f\ to\ g\ AND\ prefer\ g\ to\ h\ \[AlignmentMarker] \[DoubleRightArrow] prefer\ f\ to\ h\), "\n", \(TraditionalForm\`Pr(f, g)\[AlignmentMarker] = \(\[ScriptCapitalF]\_1\)( Pr(f), Pr(g | f))\), "\n", \(TraditionalForm\`Pf( f)\[AlignmentMarker] = \(\(\(\[ScriptCapitalF]\_2\)( Pr(\(~\)\(f\)))\)\(.\)\)\)}], "NumberedEquation", TextAlignment->AlignmentMarker, CellTags->{"Eq:ConsistencyConditions", "Ed:Change6"}], Cell[TextData[{ "In ", ButtonBox["equation", ButtonData:>"Eq:ConsistencyConditions", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConsistencyConditions"], ") ", Cell[BoxData[ \(TraditionalForm\`\[ScriptCapitalF]\_1\)]], " and ", Cell[BoxData[ \(TraditionalForm\`\[ScriptCapitalF]\_2\)]], " are yet to be determined functions. These deceptively simple consistency \ conditions are all that is required, because they lead to Cox's theorems (", ButtonBox["Cox 1946", ButtonData:>"Ref:Cox1946", ButtonStyle->"Hyperlink"], ") which obliges us to reason in terms of Bayesian calculus" }], "Text"], Cell[BoxData[ \(TraditionalForm\`Pr(f) + Pr(\(~\)\(f\))\[AlignmentMarker] = 1, \ \ \ \ \ 0 \[LessEqual] Pr(f) \[LessEqual] 1, \n Pr(falsity)\[AlignmentMarker] = 0, \n Pr(certainty)\[AlignmentMarker] = 1, \n Pr(f, g)\[AlignmentMarker] = \(Pr( f)\) \(\(Pr(g | f)\)\(.\)\)\)], "NumberedEquation", TextAlignment->AlignmentMarker, CellTags->"Eq:BayesianCalculus"], Cell["\<\ This axiomic approach to constructing and interpreting probabilities has \ become known as the Bayesian approach. A Bayesian uses probabilities to \ describe incomplete information (rather than randomness), and has an \ epistemological (rather than ontological) view of the universe. On the other \ hand, a frequentist uses a probability to describe randomness. Unnecessary \ arguments between Bayesians and frequentists arise because of their different \ interpretations of probability.\ \>", "Text"], Cell[TextData[{ "We shall usually be concerned with probabilities of continuous \ vector-valued quantities, so we shall speak of p.d.f.s. Thus define ", Cell[BoxData[ \(TraditionalForm\`Pr(Y | X)\)]], " as the p.d.f. of ", Cell[BoxData[ \(TraditionalForm\`Y\)]], " conditioned on ", Cell[BoxData[ \(TraditionalForm\`X\)]], ". We may use ", Cell[BoxData[ \(TraditionalForm\`Pr(Y | X)\)]], " to proceed deductively from ", Cell[BoxData[ \(TraditionalForm\`X\)]], " to ", Cell[BoxData[ \(TraditionalForm\`Y\)]], ". Bayes' theorem then gives ", Cell[BoxData[ \(TraditionalForm\`Pr(X | Y)\)]], " in the form" }], "Text"], Cell[BoxData[ \(TraditionalForm\`\(\(Pr( X | Y) = \(\(Pr(Y | X)\) \(Pr(X)\)\)\/\(\[Sum]\_\(X\^\[Prime]\)\(Pr(Y \ | X\^\[Prime])\) \(Pr(X\^\[Prime])\)\)\)\(,\)\)\)], "NumberedEquation", CellTags->"Eq:BayesTheorem"], Cell[TextData[{ "which we use to proceed inductively from ", Cell[BoxData[ \(TraditionalForm\`Y\)]], " to ", Cell[BoxData[ \(TraditionalForm\`X\)]], ". The most important feature about ", ButtonBox["equation", ButtonData:>"Eq:BayesTheorem", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:BayesTheorem"], ") is that ", Cell[BoxData[ \(TraditionalForm\`Pr(X | Y)\)]], " depends on an additional factor ", Cell[BoxData[ \(TraditionalForm\`Pr(X)\)]], ", the prior p.d.f. of ", Cell[BoxData[ \(TraditionalForm\`X\)]], ". Thus induction is influenced by prior expectations." }], "Text"] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " A coherent scattering and imaging model" }], "Subsection"], Cell[TextData[{ "The model which we shall use has been presented before (", ButtonBox["Luttrell 1985a,", ButtonData:>"Ref:Luttrell1985a", ButtonStyle->"Hyperlink"], " ", ButtonBox["1989", ButtonData:>"Ref:Luttrell1989", ButtonStyle->"Hyperlink"], "). Let us denote the scattering cross-section (or the object) as ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], ", the scattered field (or the object field) as ", Cell[BoxData[ \(TraditionalForm\`f\)]], ", and the data as ", Cell[BoxData[ \(TraditionalForm\`g\)]], ". We shall assume coherent illumination, in which case a commonly-used \ physical scattering model is a random walk process in which a large number of \ scattering centres (in the vicinity of the field point) each contribute \ randomly-phased pieces of scattered field, leading to a Gaussian p.d.f. for \ the total local scattered field" }], "Text"], Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma]) = \(1\/\(det\ \[Pi]\[Sigma]\)\) exp\ \(\((\(-f\^\[Dagger]\) \(\[Sigma]\^\(-1\)\) f)\)\(.\)\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ScatteringPDF"], Cell[TextData[{ "From the point of view of interpretations of probability this is a ", StyleBox["frequentist", FontSlant->"Italic"], " model. We represent the cross-section as a matrix-valued quantity (in \ this case a field-field covariance), because this permits us to model \ situations where distinct scatterers produce correlated scattered fields. The \ covariance (i.e. ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], ") of ", Cell[BoxData[ \(TraditionalForm\`f\)]], " is denoted as ", Cell[BoxData[ \(TraditionalForm\`\(\(\(<\)\(f\ f\^\[Dagger]\)\)\( > \_\(Pr( f | \[Sigma])\)\)\)\)]], " where ", Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(\[CenterEllipsis]\)\(>\)\)\)]], " denotes an average, and the p.d.f. used in the averaging process is \ indicated outside the ", Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(\[CenterEllipsis]\)\(>\)\)\)]], ". We shall restrict our attention to diagonal ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " given by ", Cell[BoxData[ \(TraditionalForm\`\[Sigma] = diag(\[Sigma]\_1, \[Sigma]\_2, \[CenterEllipsis])\)]], " (uncorrelated). This models a conventional uncorrelated scattering \ process where the matrix nature of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " is physically superfluous, but mathematically convenient." }], "Text"], Cell[TextData[{ "An alternative derivation of ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ScatteringPDF"], ") may be obtained by using maximum entropy techniques. There is an \ axiomatic derivation of this approach given in ", ButtonBox["Shore and Johnson (1980)", ButtonData:>"Ref:ShoreJohnson1980", ButtonStyle->"Hyperlink"], " which is based on consistency requirements, and there is a simplified \ version of their arguments given in ", ButtonBox["Skilling (1989)", ButtonData:>"Ref:Skilling1989", ButtonStyle->"Hyperlink"], ". This axiomatic approach does not rely on the frequency interpretation of \ the entropy of a system being the logarithm of the effective number of states \ available to that system. Thus suppose that the only knowledge we have \ concerning the form of ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], " is expressed as:" }], "Text"], Cell[TextData[{ "\t(1) ", Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(f\)\( > \_\(Pr(f | \[Sigma])\)\)\) = 0\ \((mean)\)\)]], "\n\t(2) ", Cell[BoxData[ \(TraditionalForm\`\(\(\(<\)\(f\ f\^\[Dagger]\)\)\( > \_\(Pr( f | \[Sigma])\)\)\) = \[Sigma]\ \((covariance)\)\)]], "." }], "Text"], Cell[TextData[{ "The maximum entropy p.d.f. consistent with these two constraints is the \ same as was given in ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ScatteringPDF"], "). The maximum entropy method guarantees that ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], " is the least committal (", ButtonBox["Jaynes 1968", ButtonData:>"Ref:Jaynes1968", ButtonStyle->"Hyperlink"], ") p.d.f. consistent with the constraints, so we can be certain that no \ spurious information is encoded in the p.d.f. In other words, ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], " expresses precisely our degree of ignorance about ", Cell[BoxData[ \(TraditionalForm\`f\)]], " given ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " - this is how one should construct degrees of reasonable belief given \ knowledge expressed as a set of constraints. In this approach we do not need \ to introduce a frequentist interpretation of probability, so the p.d.f. is \ the type of object which a Bayesian prefers to use." }], "Text"], Cell[TextData[{ "The Gaussian scattering model does not entirely express our knowledge of \ the physical situation. In regions where there are localised \ strongly-scattering centres (e.g. targets) the field scattered by a single \ scatterer might dominate the scattered amplitude, thus invalidating the \ simple random walk approach. However, any such additional information about \ the form of ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], " would lead to a less committal maximum entropy p.d.f., so we err on the \ side of caution if we use ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ScatteringPDF"], ") instead. We shall adopt the cautious approach of ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ScatteringPDF"], ") because the simplicity of the data processing scheme which eventually \ emerges justifies this simplification." }], "Text"], Cell[TextData[{ "We shall model the imaging process using a linear operator ", Cell[BoxData[ \(TraditionalForm\`T\)]], " (this models the system point spread function (PSF)), and we shall assume \ that the data ", Cell[BoxData[ \(TraditionalForm\`g\)]], " is corrupted by zero mean Gaussian noise ", Cell[BoxData[ \(TraditionalForm\`n\)]], " with covariance matrix ", Cell[BoxData[ \(TraditionalForm\`N\)]], ". Thus" }], "Text"], Cell[BoxData[ \(TraditionalForm\`g = T\ f + n\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ImagingModel"], Cell["which is described by the p.d.f.", "Text"], Cell[BoxData[ FormBox[ RowBox[{\(Pr(g | f)\), "=", RowBox[{ FractionBox["1", RowBox[{"det", " ", "\[Pi]", " ", StyleBox["N", FontWeight->"Bold"]}]], RowBox[{ RowBox[{"exp", " ", "[", RowBox[{\(-\((g - T\ f)\)\^\[Dagger]\), RowBox[{ SuperscriptBox[ StyleBox["N", FontWeight->"Bold"], \(-1\)], "(", \(g - T\ f\), ")"}]}], "]"}], "."}]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ImagingModelPDF"], Cell[TextData[{ "We shall restrict our attention to isotropic ", Cell[BoxData[ FormBox[ StyleBox["N", FontWeight->"Bold"], TraditionalForm]]], " (i.e. white noise) with a variance ", Cell[BoxData[ \(TraditionalForm\`\[Nu]\)]], " given by ", Cell[BoxData[ FormBox[ RowBox[{ StyleBox["N", FontWeight->"Bold"], "=", RowBox[{"\[Nu]", " ", StyleBox["I", FontWeight->"Bold"]}]}], TraditionalForm]]], ". This frequentist approach to noise modelling can be replaced by a \ maximum entropy approach, similar to that used to derive ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ScatteringPDF"], ") and leading to the same result as ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ImagingModelPDF"], ")." }], "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", " The inverse object field problem" }], "Section", CellTags->"Sect:InverseObjectFieldProblem"], Cell[TextData[{ "The object of this section is to use Bayes' theorem to obtain the \ posterior for the scattered field given the data ", Cell[BoxData[ \(TraditionalForm\`g\)]], " and the prior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], ". We thus use Bayes' theorem in the following form:" }], "Text"], Cell[BoxData[ \(TraditionalForm\`Pr(g, f, \[Sigma]) = \(\(Pr(f | g, \[Sigma])\) \(Pr(g, \[Sigma])\) = \(Pr( g | f, \[Sigma])\) \(Pr(f, \[Sigma])\)\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:JointPDF"], Cell[BoxData[ FormBox[ RowBox[{\(Pr(f | g, \[Sigma])\), "=", RowBox[{\(\(\(Pr(g | f)\) \(Pr(f | \[Sigma])\)\)\/\(Pr( g | \[Sigma])\)\), "=", FractionBox[\(\(Pr(g | f)\) \(Pr(f | \[Sigma])\)\), RowBox[{"\[Integral]", RowBox[{ StyleBox[ SuperscriptBox["df", StyleBox["\[Prime]", FontSlant->"Plain"]], FontSlant->"Italic"], \(Pr(g | f\^\[Prime])\), \(Pr( f\^\[Prime] | \[Sigma])\)}]}]]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF"], Cell[TextData[{ "where we have used the Markov chain property of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\[LongRightArrow]f\[LongRightArrow]g\)]], " to obtain ", Cell[BoxData[ \(TraditionalForm\`Pr(g | f, \[Sigma]) = Pr(g | f)\)]], " (i.e. ", Cell[BoxData[ \(TraditionalForm\`g\)]], " is conditionally independent of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " given that ", Cell[BoxData[ \(TraditionalForm\`f\)]], " is known)." }], "Text"], Cell[TextData[{ "The ", Cell[BoxData[ \(TraditionalForm\`f\)]], " dependence of ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF"], ") is contained entirely in the numerator. The posterior reduces to \ essentially the product of the prior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], " and the data dependent part ", Cell[BoxData[ \(TraditionalForm\`Pr(g | f)\)]], ". Thus Bayes' theorem provides a rigorous means of combining the prior \ knowledge with the new knowledge acquired from a dataset ", Cell[BoxData[ \(TraditionalForm\`g\)]], ". If the data is very noisy then ", Cell[BoxData[ \(TraditionalForm\`Pr(g | f)\)]], " will depend only weakly on ", Cell[BoxData[ \(TraditionalForm\`f\)]], ", and the prior will dominate the behaviour of the posterior. This is \ exactly as expected because noisy data contains little information, so our \ inference must default to the prior data." }], "Text"], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " Minimum mean square error reconstruction, ", Cell[BoxData[ \(TraditionalForm\`f\_rec\)]] }], "Subsection"], Cell[TextData[{ "We shall now derive the explicit form of ", Cell[BoxData[ \(TraditionalForm\`Pr(f | g, \[Sigma])\)]], ". We may now substitute ", ButtonBox["equation", ButtonData:>"Eq:ScatteringPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ScatteringPDF"], ") and ", ButtonBox["equation", ButtonData:>"Eq:ImagingModelPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ImagingModelPDF"], ") into ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF"], ") to obtain, after some algebra" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{ RowBox[{\(Pr(f | g, \[Sigma])\), "=", RowBox[{ FractionBox["1", RowBox[{"det", " ", "\[Pi]", " ", StyleBox["C", FontWeight->"Bold"]}]], RowBox[{"exp", " ", "[", RowBox[{\(-\((\(\(f\)\(-\)\) < f\( > \_\(Pr(f | g, \ \[Sigma])\)\))\)\^\[Dagger]\), RowBox[{ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], "(", \(\(\(f\)\(-\)\) < f\( > \_\(Pr(f | g, \[Sigma])\)\)\), ")"}]}], "]"}]}]}], ","}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF2"], Cell["whose mean and covariance are given by", "Text"], Cell[BoxData[ FormBox[ RowBox[{ RowBox[{\(\(<\)\(f\)\( > \_\(Pr(f | g, \[Sigma])\)\)\), "\[AlignmentMarker]", "=", RowBox[{ RowBox[{ StyleBox["C", FontWeight->"Bold"], " ", \(T\^\[Dagger]\), SuperscriptBox[ StyleBox["N", FontWeight->"Bold"], \(-1\)], "g"}], "=", RowBox[{"\[Sigma]", " ", \(T\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}]}]}], ",", "\n", RowBox[{\(\(<\)\(f\ f\^\[Dagger]\)\( > \_\(Pr( f | g, \[Sigma])\)\)\(-\(\(<\)\(f\)\( > \_\(Pr( f | g, \[Sigma])\)\)\(<\)\(f\^\[Dagger]\)\( > \_\(Pr( f | g, \[Sigma])\)\)\)\)\), "\[AlignmentMarker]", "=", StyleBox["C", FontWeight->"Bold"]}], ","}], TraditionalForm]], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation"], Cell[TextData[{ " where we have defined ", Cell[BoxData[ FormBox[ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], TraditionalForm]]], " and ", Cell[BoxData[ FormBox[ StyleBox["M", FontWeight->"Bold"], TraditionalForm]]], " as" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{ RowBox[{ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], "\[AlignmentMarker]", "\[Congruent]", RowBox[{ RowBox[{\(T\^\[Dagger]\), SuperscriptBox[ StyleBox["N", FontWeight->"Bold"], \(-1\)], "T"}], "+", \(\[Sigma]\^\(-1\)\)}]}], ",", "\n", RowBox[{ StyleBox["M", FontWeight->"Bold"], StyleBox["\[AlignmentMarker]", FontWeight->"Bold"], "\[Congruent]", RowBox[{\(T\ \[Sigma]\ T\^\[Dagger]\), "+", RowBox[{ StyleBox["N", FontWeight->"Bold"], StyleBox[".", FontWeight->"Plain"]}]}]}]}], TraditionalForm]], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation", CellTags->"Eq:Covariance"], Cell[TextData[{ "The posterior mean ", Cell[BoxData[ FormBox[ RowBox[{"\[Sigma]", " ", \(T\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}], TraditionalForm]]], " is non-zero due to the effect of the ", Cell[BoxData[ \(TraditionalForm\`Pr(g | f)\)]], " term in ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF"], "). The posterior covariance ", Cell[BoxData[ FormBox[ StyleBox["C", FontWeight->"Bold"], TraditionalForm]]], " is not the same as the prior covariance ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " for the same reason." }], "Text"], Cell[TextData[{ "For practical purposes a single representative ", Cell[BoxData[ \(TraditionalForm\`f\)]], " is usually chosen as the outcome of this inference process. The natural \ choice is" }], "Text"], Cell[BoxData[ \(TraditionalForm\`\(\(f\_rec = \(\(<\)\(f\)\( > \_\(Pr( f | g, \[Sigma])\)\)\)\)\(\[AlignmentMarker]\)\(,\)\)\)], \ "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Frec"], Cell[TextData[{ "where the subscript 'rec' denotes the reconstructed scattered field. This \ definition of ", Cell[BoxData[ \(TraditionalForm\`f\_rec\)]], " locates it at the peak of the posterior." }], "Text"], Cell[TextData[{ "Note that ", Cell[BoxData[ \(TraditionalForm\`f\_rec\)]], " is generated from ", Cell[BoxData[ \(TraditionalForm\`g\)]], " by applying a linear operator ", Cell[BoxData[ FormBox[ RowBox[{"\[Sigma]", " ", \(T\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)]}], TraditionalForm]]], ". In an alternative approach to obtaining a representative reconstruction \ of the scattered field we could find the linear operator which filters ", Cell[BoxData[ \(TraditionalForm\`g\)]], " in such a way as to recover the MMSE reconstruction. This is the approach \ of Wiener which leads to precisely the same linear filter that we have \ derived here from the Bayesian viewpoint (", ButtonBox["Luttrell 1985a", ButtonData:>"Ref:Luttrell1985a", ButtonStyle->"Hyperlink"], "). This equivalence follows from the use of mean and covariance as the \ only constrained statistics in both cases. However the Bayesian approach has \ the advantage of being explicitly related to the consistency conditions in ", ButtonBox["equation", ButtonData:>"Eq:ConsistencyConditions", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConsistencyConditions"], "), and thus is more amenable to consistent generalisation to other ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], " and ", Cell[BoxData[ \(TraditionalForm\`Pr(g | f)\)]], "." }], "Text"], Cell[TextData[{ "This reconstruction scheme depends critically on having available complex \ data ", Cell[BoxData[ \(TraditionalForm\`g\)]], " and accurate knowledge of the PSF ", Cell[BoxData[ \(TraditionalForm\`T\)]], ". Studies of how this Bayesian technique can be applied to synthetic \ aperture radar (SAR) data have been presented elsewhere: SAR data has a phase \ which can contain valuable information (", ButtonBox["Luttrell 1987a", ButtonData:>"Ref:Luttrell1987a", ButtonStyle->"Hyperlink"], "), and the PSF of the Royal Signals and Radar Establishment (RSRE) SAR has \ been estimated from 'targets of opportunity' (", ButtonBox["Luttrell 1987b", ButtonData:>"Ref:Luttrell1987b", ButtonStyle->"Hyperlink"], "). This technique may be used to super-resolve the data provided the \ following two conditions are met (", ButtonBox["Luttrell 1985a", ButtonData:>"Ref:Luttrell1985a", ButtonStyle->"Hyperlink"], "):" }], "Text"], Cell["\<\ \t(1) The image of the target must not be larger than a few resolution cells \ in size. \t(2) The signal-to-clutter ratio of (the modulus of ) the data must be of \ order of 10 or more.\ \>", "Text"], Cell["\<\ These two requirements ensure that the image of the target is a small bright \ blob.\ \>", "Text"] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " Information Channels" }], "Subsection", CellTags->"Sect:InformationChannels"], Cell[TextData[{ "We may gain a deeper understanding of ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF2"], ") by performing a singular value decomposition (SVD). The details of the \ appropriate weighted SVD have been presented (", ButtonBox["Luttrell 1989", ButtonData:>"Ref:Luttrell1989", ButtonStyle->"Hyperlink"], "), and we reproduce the essential steps in the ", ButtonBox["appendix", ButtonData:>"Sect:Appendix", ButtonStyle->"Hyperlink"], ". This SVD decomposes the linear imaging operator ", Cell[BoxData[ \(TraditionalForm\`T\)]], " into a number of channels, and thus simplifies the detailed analysis of \ what Bayes' theorem is achieving. The posterior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | g, \[Sigma])\)]], " now becomes" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{ RowBox[{\(Pr(f | g, \[Sigma])\), "=", RowBox[{ FractionBox["1", RowBox[{"det", " ", "\[Pi]", " ", StyleBox["C", FontWeight->"Bold"]}]], RowBox[{"exp", " ", "[", RowBox[{\(-f\_invis\^\[Dagger]\), SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], \(f\^\[Dagger]\)}], "]"}], RowBox[{"exp", " ", "[", RowBox[{\(-\((f\_vis - f\_rec)\)\^\[Dagger]\), RowBox[{ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], "(", \(f\_vis - f\_rec\), ")"}]}], "]"}]}]}], ","}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF3"], Cell[TextData[{ "where we have decomposed ", Cell[BoxData[ \(TraditionalForm\`f\)]], " as ", Cell[BoxData[ \(TraditionalForm\`f = \((f\_vis, f\_invis)\)\)]], ", and ", Cell[BoxData[ \(TraditionalForm\`f\_vis\)]], " (", Cell[BoxData[ \(TraditionalForm\`f\_invis\)]], ") is the subspace of ", Cell[BoxData[ \(TraditionalForm\`f\)]], " which is '(in)visible' to the data ", Cell[BoxData[ \(TraditionalForm\`g\)]], ". Such (in)visibility is a consequence of the low dimensionality of the \ data - we can recover from ", Cell[BoxData[ \(TraditionalForm\`g\)]], " up to ", Cell[BoxData[ \(TraditionalForm\`dim\ g\)]], " linear degrees of freedom of ", Cell[BoxData[ \(TraditionalForm\`f\)]], " (they are visible), all other degrees of freedom must remain unknown \ (they are invisible). In practice there may be fewer than ", Cell[BoxData[ \(TraditionalForm\`dim\ g\)]], " visible degrees of freedom because the data is oversampled - this could \ be inverted and used as a definition of oversampling. The density of visible \ degrees of freedom can be used to define a generalised resolution cell such \ that there is, on average, one visible degree of freedom per resolution \ cell." }], "Text"], Cell[TextData[{ "In ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF3", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF3"], ") we have defined ", Cell[BoxData[ \(TraditionalForm\`f\_invis\)]], " to lie in the subspace where the posterior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | g, \[Sigma])\)]], " is not influenced by the data: i.e. it depends only on the prior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], ". ", Cell[BoxData[ \(TraditionalForm\`f\_vis\)]], " then occupies the orthogonal complement subspace. This notation is \ explained in detail in the ", ButtonBox["appendix", ButtonData:>"Sect:Appendix", ButtonStyle->"Hyperlink"], ". The change in (inverse) covariance matrix ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\^\(-1\) \[RightArrow] \[Sigma]\^\(-1\) + \ \(T\^\[Dagger]\) \(N\^\(-1\)\) T\)]], " in passing from ", Cell[BoxData[ \(TraditionalForm\`f\_invis\)]], " to ", Cell[BoxData[ \(TraditionalForm\`f\_vis\)]], " causes the visible degrees of freedom to be more tightly constrained \ because ", Cell[BoxData[ \(TraditionalForm\`\(T\^\[Dagger]\) \(N\^\(-1\)\) T\)]], " is a non-negative definite." }], "Text"], Cell[TextData[{ "Referring to the ", ButtonBox["appendix", ButtonData:>"Sect:Appendix", ButtonStyle->"Hyperlink"], ", an interesting special case is recovered in the limit ", Cell[BoxData[ \(TraditionalForm\`\[Lambda]\_m \[GreaterGreater] \[Nu]\)]], " (signal energy much greater than noise energy in channel ", Cell[BoxData[ \(TraditionalForm\`m\)]], "), because then ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF3", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF3"], ") contains one or more hard constraints on ", Cell[BoxData[ \(TraditionalForm\`f\_vis\)]], ". These hard constraints arise because effectively ", Cell[BoxData[ FormBox[ RowBox[{ StyleBox["N", FontWeight->"Bold"], "\[TildeTilde]", StyleBox["0", FontWeight->"Bold"]}], TraditionalForm]]], ", so ", Cell[BoxData[ FormBox[ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], TraditionalForm]]], " will contain some very large eigenvalues (assuming that ", Cell[BoxData[ \(TraditionalForm\`T\)]], " is non-trivial), which in turn implies that the posterior has \ infinitesimal thickness in certain directions. This leads to ", Cell[BoxData[ \(TraditionalForm\`f\_m\^\[Prime] \[TildeTilde] g\_m\^\[Prime]\/\@\[Lambda]\_m\)]], " with a variance ", Cell[BoxData[ \(TraditionalForm\`\[Nu]\/\[Lambda]\_m\)]], ". This approximation is precisely what is used in conventional \ non-Bayesian SVD reconstruction methods which reconstruct ", Cell[BoxData[ \(TraditionalForm\`f\)]], " using ", Cell[BoxData[ \(TraditionalForm\`f\_rec = \(\@\[Sigma]\) \(\[Sum]\_\(m = 0\)\%m\_0\( \ g\_m\^\[Prime]\/\@\[Lambda]\_m\) u\^m\)\)]], " where the ", Cell[BoxData[ \(TraditionalForm\`\[Lambda]\_m\)]], " are arranged in order of decreasing size, and ", Cell[BoxData[ \(TraditionalForm\`m\_0\)]], " is chosen as large as possible but consistent with ", Cell[BoxData[ \(TraditionalForm\`\[Lambda]\_\(m\_0\) \[GreaterGreater] \[Nu]\)]], " (", ButtonBox["Bertero and Pike 1982", ButtonData:>"Ref:BerteroPike1982", ButtonStyle->"Hyperlink"], "). This sharp cut-off is a crude form of regularisation which can produce \ results indistinguishable from the full Bayesian approach when the spectrum \ of ", Cell[BoxData[ \(TraditionalForm\`\[Lambda]\_m\)]], " (ordered according to size) indeed cuts off rapidly. The Bayesian \ approach provides an interpolation between the ", Cell[BoxData[ \(TraditionalForm\`\[Lambda]\_m \[GreaterGreater] \[Nu]\)]], " and ", Cell[BoxData[ \(TraditionalForm\`\[Lambda]\_m \[LessLess] \[Nu]\)]], " regimes based on the fundamental consistency requirements of ", ButtonBox["equation", ButtonData:>"Eq:ConsistencyConditions", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConsistencyConditions"], ") ." }], "Text"], Cell[TextData[{ "As anticipated in the title of this ", ButtonBox["section", ButtonData:>"Sect:InformationChannels", ButtonStyle->"Hyperlink"], " we have performed a decomposition of the imaging system into information \ channels labelled by ", Cell[BoxData[ \(TraditionalForm\`m\)]], ". An important feature is the prior knowledge dependence of these channels \ (", ButtonBox["Luttrell and Oliver 1986", ButtonData:>"Ref:LuttrellOliver1986", ButtonStyle->"Hyperlink"], ", ", ButtonBox["1988", ButtonData:>"Ref:LuttrellOliver1988", ButtonStyle->"Hyperlink"], "): ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " plays a central r\[OHat]le in determining these channels. The \ interpretation of what we mean by ", StyleBox["information", FontSlant->"Italic"], " is delicate, so we shall give a detailed theoretical information \ treatment in ", ButtonBox["section", ButtonData:>"Sect:MutualInformation", ButtonStyle->"Hyperlink"], " ", CounterBox["Section"], ".", CounterBox["Subsection", "Sect:MutualInformation"], ". Intuitively the reason for the prior knowledge dependence of the \ information channel structure is obvious, because a channel can carry ", StyleBox["useful", FontSlant->"Italic"], " information about ", Cell[BoxData[ \(TraditionalForm\`f\)]], " only if we have not already received such information from our prior \ knowledge, by definition." }], "Text"] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " Mutual Information" }], "Subsection", CellTags->"Sect:MutualInformation"], Cell[TextData[{ "We shall now make our notion of useful information rigorous (", ButtonBox["Luttrell 1985b", ButtonData:>"Ref:Luttrell1985b", ButtonStyle->"Hyperlink"], "). Mutual information is a theoretical information measure that is used to \ quantify the degree of mutual dependence of variables. It is an objective \ information measure, because it does not depend on the ordering of the state \ spaces of the variables. Thus mutual information is a good candidate for a \ useful information measure, because it allows us to quantify the extent to \ which knowledge (expressed as a p.d.f.) of the state of one variable \ constrains our knowledge of another variable." }], "Text"], Cell[TextData[{ "Define the entropy ", Cell[BoxData[ \(TraditionalForm\`H[r]\)]], " of a variable ", Cell[BoxData[ \(TraditionalForm\`r\)]], " as" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{\(H[r]\), "=", RowBox[{"-", RowBox[{"\[Integral]", " ", RowBox[{\(Pr(r)\), \(log\ [Pr(r)]\), StyleBox["dr", FontSlant->"Italic"]}]}]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "An explanation of the use of entropy and related quantities in information \ theory can be found in ", ButtonBox["Shannon and Weaver (1949)", ButtonData:>"Ref:ShannonWeaver1949", ButtonStyle->"Hyperlink"], ". We shall paraphrase their interpretation of entropy in frequentist \ language because it provides an intuitive operational definition of entropy." }], "Text"], Cell[TextData[{ "\t(1) Draw ", Cell[BoxData[ \(TraditionalForm\`s\)]], " samples independently from ", Cell[BoxData[ \(TraditionalForm\`Pr(r)\)]], " to form the ordered set ", Cell[BoxData[ \(TraditionalForm\`R\)]], ", ", Cell[BoxData[ \(TraditionalForm\`R \[Congruent] {r\_1, r\_2, \[CenterEllipsis], r\_s}\)]], ".\n\t(2) Form the set of all such ordered sets ", Cell[BoxData[ \(TraditionalForm\`\[ScriptCapitalR]\)]], ", ", Cell[BoxData[ \(TraditionalForm\`\[ScriptCapitalR] \[Congruent] {R : \ all\ distinct\ sets\ of\ samples}\)]], ".\n\t(3) As ", Cell[BoxData[ \(TraditionalForm\`s\[LongRightArrow]\[Infinity]\)]], " we find that ", Cell[BoxData[ \(TraditionalForm\`s\ H[r]\)]], " measures the logarithm of the number of sets ", Cell[BoxData[ \(TraditionalForm\`R\)]], " contained in ", Cell[BoxData[ \(TraditionalForm\`\[ScriptCapitalR]\)]], " which have a high probability of occurring (i.e. are not outliers).\n\t\ (4) There are thus ", Cell[BoxData[ \(TraditionalForm\`s\ H[r]\)]], " distinct 'messages' of length ", Cell[BoxData[ \(TraditionalForm\`s\)]], " which are produced by the memoryless 'source', which may be used for \ communicating information." }], "Text"], Cell[TextData[{ "The interpretation of ", Cell[BoxData[ \(TraditionalForm\`H[r]\)]], " as the logarithm of the effective number of states available to a single \ sample ", Cell[BoxData[ \(TraditionalForm\`r\)]], " is now apparent." }], "Text"], Cell[TextData[{ "The mutual information ", Cell[BoxData[ \(TraditionalForm\`I[f; g]\)]], " between a pair of variables ", Cell[BoxData[ \(TraditionalForm\`f\)]], " and ", Cell[BoxData[ \(TraditionalForm\`g\)]], " is defined in three equivalent ways as" }], "Text"], Cell[BoxData[ \(TraditionalForm\`I[f; g] = H[f] + H[g] - H[f, g]\)], "NumberedEquation",\ CounterIncrements->"NumberedEquation"], Cell[BoxData[ \(TraditionalForm\`\(\(\ \ \ \ \ \ \ \ \ \ \ \ \ \ \)\(\(=\)\(H[f] - H[f | g]\)\)\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation2"], Cell[BoxData[ \(TraditionalForm\`\(\(\[AlignmentMarker]\ \ \ \ \ \ \ \ \ \ \ \ \ \ \)\(\ \(=\)\(H[g] - H[g | f]\)\)\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation3"], Cell[TextData[{ "In the frequency interpretation, each of these expressions is a difference \ of entropies which measures the logarithm of ", StyleBox["the ratio of", FontSlant->"Italic"], " the effective number of states available jointly to ", Cell[BoxData[ \(TraditionalForm\`f\)]], " and ", Cell[BoxData[ \(TraditionalForm\`g\)]], ", firstly considered as independent variables then considered as dependent \ variables. Thus ", Cell[BoxData[ \(TraditionalForm\`I[f; g]\)]], " qualifies as an objective measure of the mutual dependence of ", Cell[BoxData[ \(TraditionalForm\`f\)]], " and ", Cell[BoxData[ \(TraditionalForm\`g\)]], "." }], "Text"], Cell[TextData[{ "Starting from ", ButtonBox["equation", ButtonData:>"Eq:MutualInformation2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:MutualInformation2"], ") or ", ButtonBox["equation", ButtonData:>"Eq:MutualInformation3", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:MutualInformation3"], ") we obtain, respectively" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{\(I[f; g]\), "\[AlignmentMarker]", "=", RowBox[{ RowBox[{"log", " ", "[", FractionBox[\(det\ \[Sigma]\), RowBox[{"det", " ", StyleBox["C", FontWeight->"Bold"]}]], "]"}], "=", RowBox[{"log", " ", "[", FractionBox[ RowBox[{"det", " ", StyleBox["M", FontWeight->"Bold"]}], RowBox[{"det", " ", StyleBox["N", FontWeight->"Bold"]}]], "]"}]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation4"], Cell[TextData[{ "where we have used the definitions in ", ButtonBox["equation", ButtonData:>"Eq:Covariance", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Covariance"], "). A determinant of a covariance matrix measures the effective (squared) \ volume occupied by the peak of the corresponding multidimensional complex \ Gaussian p.d.f., i.e. the product of the eigenvalues (or squared widths) of \ its covariance matrix. Thus ", Cell[BoxData[ \(TraditionalForm\`det\ \((\[Sigma])\)\)]], " measures the squared volume of the prior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], ", and ", Cell[BoxData[ FormBox[ RowBox[{"det", " ", RowBox[{"(", StyleBox["C", FontWeight->"Bold"], ")"}]}], TraditionalForm]]], " measures the squared volume of the posterior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | g, \[Sigma])\)]], ". The ratio of these is the factor by which the squared volume of the \ prior is 'squeezed' by the data in order to become the posterior. Similarly, \ ", Cell[BoxData[ FormBox[ RowBox[{"det", " ", RowBox[{"(", StyleBox["M", FontWeight->"Bold"], ")"}]}], TraditionalForm]]], " measures the squared volume of the overall (i.e. averaged over all \ scattered field configurations) data p.d.f. ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], " where" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{\(Pr(g | \[Sigma])\), "=", RowBox[{ RowBox[{"\[Integral]", RowBox[{ StyleBox["df", FontSlant->"Italic"], " ", \(Pr(g | f)\), \(Pr(f | \[Sigma])\)}]}], "=", RowBox[{ FractionBox["1", RowBox[{"det", " ", "\[Pi]", " ", StyleBox["M", FontWeight->"Bold"]}]], RowBox[{"exp", " ", "[", RowBox[{\(-g\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}], "]"}]}]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:DataPDF"], Cell[TextData[{ "and ", Cell[BoxData[ FormBox[ RowBox[{"det", " ", RowBox[{"(", StyleBox["N", FontWeight->"Bold"], ")"}]}], TraditionalForm]]], " measures the squared volume of the p.d.f. of the additive data noise, so \ the ratio of these is the number of times the noise p.d.f. can fit inside the \ overall data p.d.f. Both of these are clearly information measures, and they \ are equal to each other according to ", ButtonBox["equation", ButtonData:>"Eq:MutualInformation4", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:MutualInformation4"], ")." }], "Text"], Cell[TextData[{ "We may decompose ", Cell[BoxData[ \(TraditionalForm\`\(\(I[f; g]\)\(\[AlignmentMarker]\)\)\)]], " into components by applying the SVD of the ", ButtonBox["appendix", ButtonData:>"Sect:Appendix", ButtonStyle->"Hyperlink"], " to yield" }], "Text"], Cell[BoxData[ \(TraditionalForm\`I[f; g]\[AlignmentMarker] = \[Sum]\+m log\ [\(\[Lambda]\_m + \[Nu]\)\/\[Nu]]\)], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "This decomposition makes it clear that the SVD (as presented in the ", ButtonBox["appendix", ButtonData:>"Sect:Appendix", ButtonStyle->"Hyperlink"], ") decomposes the imaging system into information channels each of which \ contributes ", StyleBox["independently", FontSlant->"Italic"], " ", Cell[BoxData[ \(TraditionalForm\`log\ [\((\[Lambda]\_m/\[Nu] + 1)\)]\)]], " to the overall mutual information ", Cell[BoxData[ \(TraditionalForm\`I[f; g]\)]], ". What we informally called information channels in ", ButtonBox["section", ButtonData:>"Sect:InformationChannels", ButtonStyle->"Hyperlink"], " ", CounterBox["Section", "Sect:InformationChannels"], ".", CounterBox["Subsection", "Sect:InformationChannels"], " may therefore now be raised to the status of true information channels, \ in the technical sense of information as defined in information theory." }], "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", " The inverse cross-section problem" }], "Section", CellTags->"Sect:InverseCrossSectionProblem"], Cell[TextData[{ "In ", ButtonBox["section", ButtonData:>"Sect:InverseObjectFieldProblem", ButtonStyle->"Hyperlink"], " ", CounterBox["Section", "Sect:InverseObjectFieldProblem"], " we studied a Bayesian solution to the inverse field problem. Recovery of \ the scattered field is almost universally the favoured approach in \ super-resolution theory. However, this is fundamentally flawed because image \ interpretation should not be expressed in terms of the object (in this case \ the scattered field) which ", StyleBox["transports", FontSlant->"Italic"], " information, rather we should express ourselves in terms of the objects \ (in this case scattering centres) which are the ", StyleBox["source", FontSlant->"Italic"], " of the information. We should therefore be concerned with the underlying \ scattering cross-section, so here we shall present some results which are \ required in the Bayesian analysis of the inverse cross-section problem." }], "Text"], Cell["\<\ We shall now use a form of Bayes' theorem which is appropriate to this new \ inverse problem,\ \>", "Text"], Cell[BoxData[ FormBox[ RowBox[{\(Pr(\[Sigma] | g)\), "=", FractionBox[\(\(Pr(g | \[Sigma])\) \(Pr(\[Sigma])\)\), RowBox[{"\[Integral]", RowBox[{ SuperscriptBox[ StyleBox[ RowBox[{ StyleBox["d", FontSlant->"Italic"], "\[Sigma]"}]], "\[Prime]"], \(Pr( g | \[Sigma]\^\[Prime])\), \(Pr(\[Sigma]\^\[Prime])\)}]}]]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "where ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], " is the overall data p.d.f. given in ", ButtonBox["equation", ButtonData:>"Eq:DataPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:DataPDF"], "), and ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma])\)]], " is a ", StyleBox["prior", FontSlant->"Italic"], " which specifies our prior knowledge of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], "." }], "Text"], Cell[TextData[{ "The ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " dependence of ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma] | g)\)]], " comes from both ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma])\)]], " and the matrix ", Cell[BoxData[ FormBox[ StyleBox["M", FontWeight->"Bold"], TraditionalForm]]], " in ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], ", and unfortunately does not reduce to a Gaussian behaviour amenable to \ linear techniques. We shall postpone the issue of prior knowledge and the ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma])\)]], " term for now, in order to concentrate on the structure of the data \ collection term ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], ". This term is nonlinear in ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], ", so we shall develop its Taylor expansion in powers of \[Sigma]. In order \ to do this we need to differentiate ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], " with respect to ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], ", and the next two sections are concerned with this task." }], "Text"], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " First derivative of ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]] }], "Subsection", CellTags->"Sect:FirstDerivative"], Cell[TextData[{ "For convenience we shall define ", Cell[BoxData[ \(TraditionalForm\`L(g | \[Sigma])\)]], " (the log likelihood ratio) as" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{\(L(g | \[Sigma]) \[Congruent] log\ [Pr(g | \[Sigma])]\), "=", RowBox[{ RowBox[{\(-g\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}], "-", RowBox[{"log", " ", "[", RowBox[{"det", " ", "[", StyleBox["M", FontWeight->"Bold"], "]"}], "]"}], "+", \(\(constant\)\(.\)\)}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Likelihood"], Cell["In order to differentiate this we need the following results", "Text"], Cell[BoxData[{ FormBox[ RowBox[{ FractionBox[ RowBox[{"\[PartialD]", SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)]}], \(\[PartialD]\[Sigma]\_i\)], "\[AlignmentMarker]", "=", RowBox[{ RowBox[{"-", SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)]}], FractionBox[ RowBox[{"\[PartialD]", StyleBox["M", FontWeight->"Bold"]}], \(\[PartialD]\[Sigma]\_i\)], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)]}]}], TraditionalForm], "\n", FormBox[ RowBox[{ FractionBox[ RowBox[{"\[PartialD]", RowBox[{"log", " ", "[", RowBox[{"det", " ", "[", StyleBox["M", FontWeight->"Bold"], "]"}], "]"}]}], \(\[PartialD]\[Sigma]\_i\)], "\[AlignmentMarker]", "=", RowBox[{ FractionBox[ RowBox[{"\[PartialD]", RowBox[{"tr", " ", "[", RowBox[{"log", " ", "[", StyleBox["M", FontWeight->"Bold"], "]"}], "]"}]}], \(\[PartialD]\[Sigma]\_i\)], "=", RowBox[{ RowBox[{"tr", " ", "[", FractionBox[ RowBox[{"\[PartialD]", RowBox[{"log", " ", "[", StyleBox["M", FontWeight->"Bold"], "]"}]}], \(\[PartialD]\[Sigma]\_i\)], "]"}], "\n", "\[AlignmentMarker]", "=", RowBox[{"tr", " ", "[", RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], FractionBox[ RowBox[{"\[PartialD]", StyleBox["M", FontWeight->"Bold"]}], \(\[PartialD]\[Sigma]\_i\)]}], "]"}]}]}]}], TraditionalForm]}], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation", CellTags->"Eq:Derivatives"], Cell[TextData[{ "where it is important that we maintain the order of the various powers of \ ", Cell[BoxData[ FormBox[ StyleBox["M", FontWeight->"Bold"], TraditionalForm]]], " because ", Cell[BoxData[ FormBox[ StyleBox["M", FontWeight->"Bold"], TraditionalForm]]], " is an operator. Differentiating ", ButtonBox["equation", ButtonData:>"Eq:Likelihood", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Likelihood"], ") with respect to ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\_i\)]], " and using the results of ", ButtonBox["equation", ButtonData:>"Eq:Derivatives", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Derivatives"], ") yields" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{\(\[PartialD]\(L(g | \[Sigma])\)\/\[PartialD]\[Sigma]\_i\), "=", RowBox[{ RowBox[{\(g\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], FractionBox[ RowBox[{"\[PartialD]", StyleBox["M", FontWeight->"Bold"]}], \(\[PartialD]\[Sigma]\_i\)], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}], "-", RowBox[{"tr", " ", "[", RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], FractionBox[ RowBox[{"\[PartialD]", StyleBox["M", FontWeight->"Bold"]}], \(\[PartialD]\[Sigma]\_i\)]}], "]"}]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivative"], Cell["We may simplify this by using", "Text"], Cell[BoxData[ FormBox[ RowBox[{\(\[LeftBracketingBar]f\_\(rec, i\)\[RightBracketingBar]\^2\/\ \[Sigma]\_i\%2\), "=", RowBox[{ RowBox[{\(g\^\[Dagger]\), SubscriptBox[ RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], " ", "[", \(T\ T\^\[Dagger]\), "]"}], "i"], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}], "=", RowBox[{\(g\^\[Dagger]\), SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], FractionBox[ RowBox[{"\[PartialD]", StyleBox["M", FontWeight->"Bold"]}], \(\[PartialD]\[Sigma]\_i\)], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}]}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Frec2"], Cell[TextData[{ "where ", Cell[BoxData[ \(TraditionalForm\`\(\([\)\(T\ T\^\[Dagger]\)\(]\)\)\_i\)]], " is ", Cell[BoxData[ \(TraditionalForm\`T\ T\^\[Dagger]\)]], " with its internal summation restricted to position ", Cell[BoxData[ \(TraditionalForm\`i\)]], " alone. We may average ", Cell[BoxData[ \(TraditionalForm\`\[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\)]], " over data according to ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], " (i.e. for all scattered field and data noise configurations), to yield" }], "Text", CellTags->"Ed:Change1"], Cell[BoxData[ FormBox[ RowBox[{\(\(\(<\)\(\[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\)\( > \_\(Pr(g | \[Sigma])\)\)\)\/\[Sigma]\_i\%2\), "=", RowBox[{"tr", " ", "[", RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], FractionBox[ RowBox[{"\[PartialD]", StyleBox["M", FontWeight->"Bold"]}], \(\[PartialD]\[Sigma]\_i\)]}], "]"}]}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "where we have used ", Cell[BoxData[ FormBox[ RowBox[{\(\(\(<\)\(g\ g\^\[Dagger]\)\)\( > \_\(Pr(g | \[Sigma])\)\)\), "=", StyleBox["M", FontWeight->"Bold"]}], TraditionalForm]]], " from ", ButtonBox["equation", ButtonData:>"Eq:DataPDF", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:DataPDF"], "), and the cyclic property ", Cell[BoxData[ \(TraditionalForm\`tr\ \((X\ Y)\) = tr\ \((Y\ X)\)\)]], " of traces. Physically, ", Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(\[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\)\( > \_\(Pr(g | \[Sigma])\)\)\)\)]], " is the average of the intensity MMSE reconstructions over all possible \ data which arise from a given cross-section. It will therefore be rather \ smoother than any of the individual ", Cell[BoxData[ \(TraditionalForm\`\[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\)]], " in ", ButtonBox["equation", ButtonData:>"Eq:Frec2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Frec2"], "). Essentially this smoothing process is a speckle reduction technique, \ and it leads to an average intensity reconstruction which closely mimics (a \ smoothed version of ) the underlying cross-section." }], "Text"], Cell[TextData[{ ButtonBox["Equation", ButtonData:>"Eq:LikelihoodDerivative", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative"], ") now becomes (in dimensionless form)" }], "Text"], Cell[BoxData[ \(TraditionalForm\`\[PartialD]\(L(g | \[Sigma])\)\/\(\[PartialD]log\ \((\ \[Sigma]\_i)\)\) = \(\(\(\[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\)\(-\)\) < \[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\( > \_\(Pr(g | \[Sigma])\)\)\)\/\[Sigma]\_i\)], \ "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivative2"], Cell[TextData[{ "This is appealing because it indicates that we can increase the log \ likelihood by increasing ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\_i\)]], ", in those regions where the intensity reconstruction is greater than the \ smoothed intensity reconstruction (and vice versa)." }], "Text"], Cell[TextData[{ "This forms the basis of an iterated inverse cross-section algorithm in \ which an estimate of the underlying cross-section ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " is used to estimate the scattered field ", Cell[BoxData[ \(TraditionalForm\`f\_rec\)]], " using ", ButtonBox["equation", ButtonData:>"Eq:Frec", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Frec"], "), which in turn is used to update ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " using ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivative2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative2"], "). Such a scheme is heuristic because it attempts to maximise the \ likelihood function ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]], " rather than the posterior ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma] | g)\)]], ". Only ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma] | g)\)]], " includes prior knowledge ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma])\)]], " of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " as required of a Bayesian approach. In extensive numerical simulations we \ find that omitting the prior knowledge, causes ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " to 'condense out' into a number of isolated spikes if the algorithm is \ run through a sufficient number of iterations. This is a consequence of the \ uncontrolled positive feedback loop which connects ", Cell[BoxData[ \(TraditionalForm\`f\_\(rec, i\)\)]], " and ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " in the iterated algorithm: appropriate prior knowledge can temper the \ effects of this loop. Note that gradient ascent of ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma] | g)\)]], " would use" }], "Text"], Cell[BoxData[ \(TraditionalForm\`\[PartialD]log\ [Pr(\[Sigma] | g)]\/\(\[PartialD]log\ \ \((\[Sigma]\_i)\)\)\[AlignmentMarker] = \[PartialD]log\ [Pr(g | \ \[Sigma])]\/\(\[PartialD]log\ \((\[Sigma]\_i)\)\) + \[PartialD]log\ [Pr(\ \[Sigma])]\/\(\[PartialD]log\ \((\[Sigma]\_i)\)\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:PosteriorDerivative"], Cell[TextData[{ "In ", ButtonBox["equation", ButtonData:>"Eq:PosteriorDerivative", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:PosteriorDerivative"], ") the first term is given by ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivative2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative2"], ") and the second term depends on the gradient of the prior. The second \ term modifies the gradient of the likelihood function, and we can simulate \ the general effect of this without actually calculating it in detail. Thus, a \ heuristic form of prior knowledge of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " can be introduced which modifies the result in ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivative2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative2"], ") as appropriate. Such an approach has been used successfully to \ super-resolve imagery in (", ButtonBox["Luttrell and Oliver 1986", ButtonData:>"Ref:LuttrellOliver1986", ButtonStyle->"Hyperlink"], ", ", ButtonBox["1988", ButtonData:>"Ref:LuttrellOliver1988", ButtonStyle->"Hyperlink"], ")." }], "Text"] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", CounterBox["Subsection"], ".", " Higher derivatives of ", Cell[BoxData[ \(TraditionalForm\`Pr(g | \[Sigma])\)]] }], "Subsection"], Cell[TextData[{ "Simple reconstruction algorithms need only use the first derivative of ", Cell[BoxData[ \(TraditionalForm\`L(g | \[Sigma])\)]], ". However, more sophisticated update procedures can be employed if the \ second derivative matrix (Hessian) is computed in closed form. We can \ continue to differentiate ", Cell[BoxData[ \(TraditionalForm\`L(g | \[Sigma])\)]], " in order to develop higher-order terms in its Taylor expansion, but we \ shall merely summarise the main result the details can be found in (", ButtonBox["Luttrell 1989", ButtonData:>"Ref:Luttrell1989", ButtonStyle->"Hyperlink"], "). Define ", Cell[BoxData[ \(TraditionalForm\`\(D\_n\) L\)]], " as" }], "Text"], Cell[BoxData[ \(TraditionalForm\`\(D\_n\) \(L( g | \[Sigma])\) \[Congruent] \(\([\)\(\[Product]\+\(s = 1\)\%n\ \[PartialD]\/\(\[PartialD]log\ \((\[Sigma]\_\(i\_s\))\)\)\)\(]\)\) \(L( g | \[Sigma])\) \[Congruent] \ \[PartialD]\_\(i\_1\)\(\[PartialD]\_\(i\_2\)\[CenterEllipsis]\) \(\(\ \[PartialD]\_\(i\_n\)\(L(g | \[Sigma])\)\)\(.\)\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "After some algebra (", ButtonBox["Luttrell 1989", ButtonData:>"Ref:Luttrell1989", ButtonStyle->"Hyperlink"], ") we obtain" }], "Text"], Cell[BoxData[{ FormBox[\(\(D\_n\) \(L(g | \[Sigma])\)\[AlignmentMarker] = B\_n - \(\(\((\(<\)\(B\))\)\_n\)\(>\)\)\/n\), TraditionalForm], "\n", FormBox[ RowBox[{ RowBox[{\(B\_n\), "\[AlignmentMarker]", "\[Congruent]", RowBox[{\(\((\(-1\))\)\^\(n - 1\)\), RowBox[{\(\[Sum]\+perms\), RowBox[{\(g\^\[Dagger]\), RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "(", RowBox[{\(\[PartialD]\_\(p\_1\)\), StyleBox["M", FontWeight->"Bold"]}], ")"}], RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "(", RowBox[{\(\[PartialD]\_\(p\_2\)\), StyleBox["M", FontWeight->"Bold"]}], ")"}], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "\[CenterEllipsis]", " ", RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "(", RowBox[{\(\[PartialD]\_\(p\_n\)\), StyleBox["M", FontWeight->"Bold"]}], ")"}], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "g"}]}]}]}], ",", "\n", RowBox[{"<", \(B\_n\), ">", "\[AlignmentMarker]", "\[Congruent]", RowBox[{\(\((\(-1\))\)\^\(n - 1\)\), RowBox[{\(\(\[Sum]tr\)\+perms\), " ", "[", RowBox[{ RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "(", RowBox[{\(\[PartialD]\_\(p\_1\)\), StyleBox["M", FontWeight->"Bold"]}], ")"}], RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "(", RowBox[{\(\[PartialD]\_\(p\_2\)\), StyleBox["M", FontWeight->"Bold"]}], ")"}], SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "\[CenterEllipsis]", " ", RowBox[{ SuperscriptBox[ StyleBox["M", FontWeight->"Bold"], \(-1\)], "(", RowBox[{\(\[PartialD]\_\(p\_n\)\), StyleBox["M", FontWeight->"Bold"]}], ")"}]}], "]"}]}]}], ","}], TraditionalForm]}], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivativeN"], Cell[TextData[{ "where the summation is over permutations of the indices ", Cell[BoxData[ \(TraditionalForm\`\((p\_1, p\_2, \[CenterEllipsis]\ , p\_n)\)\)]], ". Note that ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivativeN", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation"], ") reduces to ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivative", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative"], ") when ", Cell[BoxData[ \(TraditionalForm\`n = 1\)]], ", as expected." }], "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[TextData[{ CounterBox["Section"], ".", " Conclusions" }], "Section"], Cell[TextData[{ "We have presented a consistent Bayesian approach to super-resolution of \ SAR data. One could simply define a Wiener filter and then perform a MMSE \ reconstruction from the data, but this would deny the wealth of \ interpretative power that is available to the Bayesian. Why should you \ minimise the squared error rather than anything else? The Bayesian approach \ rejects the naive notion of distance measures, and replaces them with \ physically and information theoretically derived priors and posteriors. In \ the special case of Gaussian p.d.f.s. this happens to reduce to a MMSE \ reconstruction prescription. In more subtle cases the advantages of Bayesian \ calculus become overwhelming, because it alone is guaranteed to provide a \ consistent approach to inference (", ButtonBox["Cox 1946", ButtonData:>"Ref:Cox1946", ButtonStyle->"Hyperlink"], ")." }], "Text"], Cell[TextData[{ "The results of ", ButtonBox["section", ButtonData:>"Sect:InverseObjectFieldProblem", ButtonStyle->"Hyperlink"], " ", CounterBox["Section", "Sect:InverseObjectFieldProblem"], " and ", ButtonBox["section", ButtonData:>"Sect:InverseCrossSectionProblem", ButtonStyle->"Hyperlink"], " ", CounterBox["Section", "Sect:InverseCrossSectionProblem"], " can be combined to produce a practical super-resolution algorithm (", ButtonBox["Luttrell and Oliver 1986", ButtonData:>"Ref:LuttrellOliver1986", ButtonStyle->"Hyperlink"], ", ", ButtonBox["1988", ButtonData:>"Ref:LuttrellOliver1988", ButtonStyle->"Hyperlink"], "). ", ButtonBox["Equation", ButtonData:>"Eq:Frec", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Frec"], ") provides the MMSE estimate of the scattered field given the data ", Cell[BoxData[ \(TraditionalForm\`g\)]], " and the prior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | \[Sigma])\)]], ". This result can be inserted into ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivative2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative2"], ") to provide the direction in which to vary ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " in order to increase the log likelihood function ", Cell[BoxData[ \(TraditionalForm\`L(g | \[Sigma])\)]], ". A smoothed version of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " can be used as an estimate of ", Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(\[LeftBracketingBar]f\_\(rec, i\)\ \[RightBracketingBar]\^2\)\( > \_\(Pr(g | \[Sigma])\)\)\)\)]], " in ", ButtonBox["equation", ButtonData:>"Eq:LikelihoodDerivative2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:LikelihoodDerivative2"], ") (see ", ButtonBox["section", ButtonData:>"Sect:FirstDerivative", ButtonStyle->"Hyperlink"], " ", CounterBox["Section", "Sect:InverseCrossSectionProblem"], ".", CounterBox["Subsection", "Sect:FirstDerivative"], "). Furthermore, by using an iterative approach, we may obtain a fast \ numerical implementation of this approach to super-resolution (", ButtonBox["Pryde et al. 1988a", ButtonData:>"Ref:PrydeDelvesLuttrell1988a", ButtonStyle->"Hyperlink"], ", ", ButtonBox["b", ButtonData:>"Ref:PrydeDelvesLuttrell1988b", ButtonStyle->"Hyperlink"], ", ", ButtonBox["c", ButtonData:>"Ref:PrydeDelvesLuttrell1988c", ButtonStyle->"Hyperlink"], ")." }], "Text"], Cell[TextData[{ "Much work remains to be done in this field. There are uncertainties about \ the stability of the PSF measured in (", ButtonBox["Luttrell 1987b", ButtonData:>"Ref:Luttrell1987b", ButtonStyle->"Hyperlink"], ") - it would be desirable to have a super-resolution method which \ simultaneously recovered both the target and PSF. However, given the limited \ data which is available, a great deal of prior knowledge about the point \ spread function would have to be introduced in order to obtain useful \ results. In the longer term the nature of the prior ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma])\)]], " needs further study, because we currently introduce prior knowedge of ", Cell[BoxData[ \(TraditionalForm\`\[Sigma]\)]], " in a somewhat heuristic fashion, rather than by rigorous Bayesian \ application of ", Cell[BoxData[ \(TraditionalForm\`Pr(\[Sigma])\)]], "." }], "Text"] }, Closed]], Cell[CellGroupData[{ Cell["Appendix: the singular value decomposition (SVD)", "Section", CellTags->"Sect:Appendix"], Cell[TextData[{ "It is very convenient to define a weighted SVD of ", Cell[BoxData[ \(TraditionalForm\`T\)]], " as follows:" }], "Text"], Cell[BoxData[ FormBox[GridBox[{ {\(\(T \(\@ \[Sigma]\) u\^m = \(\@\[Lambda]\_m\) v\^m\)\(,\)\), " ", \(\(\@\[Sigma]\) \(T\^\[Dagger]\) v\^m = \(\@\[Lambda]\_m\) u\^m\)}, {\(\(\(\(u\^\[Dagger]\)\^m\) u\^n = \[Delta]\_\(m\ n\)\)\(,\)\), " ", \(\(\(v\^\[Dagger]\)\^m\) v\^n = \[Delta]\_\(m\ n\)\)} }], TraditionalForm]], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation", CellTags->"Eq:SVD"], Cell[TextData[{ "Indices ", Cell[BoxData[ \(TraditionalForm\`m\)]], " and ", Cell[BoxData[ \(TraditionalForm\`n\)]], " label the order of the singular function (or vector). The ", Cell[BoxData[ \(TraditionalForm\`u\^m\)]], " singular ", StyleBox["functions", FontSlant->"Italic"], " are used to decompose the ", StyleBox["continuous", FontSlant->"Italic"], " scattered field. These components are mapped to the ", Cell[BoxData[ \(TraditionalForm\`v\^m\)]], " which are the corresponding ", StyleBox["discrete", FontSlant->"Italic"], " data space singular ", StyleBox["vectors", FontSlant->"Italic"], ". The singular value ", Cell[BoxData[ \(TraditionalForm\`\@\[Lambda]\_m\)]], " gives the amplitude attenuation factor in channel ", Cell[BoxData[ \(TraditionalForm\`m\)]], ", and the weight ", Cell[BoxData[ \(TraditionalForm\`\@\[Sigma]\)]], " provides an amplitude scale for the scattered field." }], "Text", CellTags->"Ed:Change2"], Cell[TextData[{ "We may use ", ButtonBox["equation", ButtonData:>"Eq:SVD", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:SVD"], ") to write" }], "Text"], Cell[BoxData[ FormBox[GridBox[{ {\(\(T \@ \[Sigma] = \[Sum]\+m\(\@ \[Lambda]\_m\) \(v\^m\) \(u\^\ \[Dagger]\)\^m\)\(,\)\), " ", \(\(\@\[Sigma]\) T\^\[Dagger] = \[Sum]\+m\(\@ \[Lambda]\_m\) \(u\^m\) \(v\^\ \[Dagger]\)\^m\)} }], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "and we shall expand ", Cell[BoxData[ \(TraditionalForm\`f\)]], " and ", Cell[BoxData[ \(TraditionalForm\`g\)]], " using" }], "Text"], Cell[BoxData[{ \(TraditionalForm\`f\[AlignmentMarker] = \(f\_invis + \(\@\[Sigma]\) \(\(\ \[Sum]\+m\(\( f\^\[Prime]\)\_m\) u\^m\)\n g\)\[AlignmentMarker] = \[Sum]\+m\(\( g\^\[Prime]\)\_m\) v\^m\)\n\(\(f\^\[Prime]\)\_m\[AlignmentMarker] \[Congruent] \ \(\(u\^\[Dagger]\)\^m\) \(1\/\@\[Sigma]\) f\)\), "\n", \(TraditionalForm\`\(\(u\^\[Dagger]\)\^m\) \(1\/\@\[Sigma]\) f\_invis\[AlignmentMarker] \[Congruent] 0\), "\n", \(TraditionalForm\`\(g\^\[Prime]\)\_m\[AlignmentMarker] \[Congruent] \ \(\(v\^\[Dagger]\)\^m\) g\)}], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation"], Cell[TextData[{ "where ", Cell[BoxData[ \(TraditionalForm\`f\_invis\)]], " is the orthogonal complement piece of ", Cell[BoxData[ \(TraditionalForm\`f\)]], " which does not project onto the ", Cell[BoxData[ \(TraditionalForm\`u\^m\)]], " (i.e. it is invisible to the data), and the ", Cell[BoxData[ \(TraditionalForm\`\(f\^\[Prime]\)\_m\)]], " constitute the expansion coefficients of the SVD. The subspace spanned by \ the SVD is all ", StyleBox["potentially", FontSlant->"Italic"], " visible to the data, provided that ", Cell[BoxData[ \(TraditionalForm\`\@\[Lambda]\_m \[GreaterGreater] \[Nu], \ \[ForAll] \ m\)]], ". Whence ", Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(f\)\( > \_\(Pr(f | g, \[Sigma])\)\)\)\)]], " in ", ButtonBox["equation", ButtonData:>"Eq:ConditionalPDF2", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:ConditionalPDF2"], ") is given by" }], "Text"], Cell[BoxData[ \(TraditionalForm\`\(\(<\)\(f\)\( > \_\(Pr( f | g, \[Sigma])\)\)\) = \(\@\[Sigma]\) \(\[Sum]\+m\(\@ \ \[Lambda]\_m\/\(\[Lambda]\_m + \[Nu]\)\) \(\(g\^\[Prime]\)\_m\) u\^m\)\)], "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Ed:Change3"], Cell[TextData[{ "and the covariance of the posterior ", Cell[BoxData[ FormBox[ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], TraditionalForm]]], " in ", ButtonBox["equation", ButtonData:>"Eq:Covariance", ButtonStyle->"Hyperlink"], " (", CounterBox["NumberedEquation", "Eq:Covariance"], ") is given by" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{ SubscriptBox[ RowBox[{"(", RowBox[{\(\@\[Sigma]\), SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], \(\@\[Sigma]\)}], ")"}], \(i\ j\)], "=", \(\[Sum]\+m\( \[Lambda]\_m\/\[Nu]\) \(u\_i\%m\) u\_j\%\(\(*\)\(\ \)\(m\)\) + \[Delta]\_\(i\ j\)\)}], TraditionalForm]], "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[TextData[{ "where the ", Cell[BoxData[ \(TraditionalForm\`i\)]], " and ", Cell[BoxData[ \(TraditionalForm\`j\)]], " suffixes are used to label object space. Thus the posterior ", Cell[BoxData[ \(TraditionalForm\`Pr(f | g, \[Sigma])\)]], " may be simplified by combining the results given in this appendix to \ yield" }], "Text"], Cell[BoxData[ FormBox[ RowBox[{ RowBox[{\(\((\(\(f\)\(-\)\) < f\( > \_\(Pr(f | g, \[Sigma])\)\))\)\^\[Dagger]\), "\[AlignmentMarker]", RowBox[{ SuperscriptBox[ StyleBox["C", FontWeight->"Bold"], \(-1\)], "(", \(\(\(f\)\(-\)\) < f\( > \_\(Pr(f | g, \[Sigma])\)\)\), ")"}]}], "=", "\[AlignmentMarker]", \(\(\(f\^\[Dagger]\)\_invis\) \ \(\[Sigma]\^\(-1\)\) f\_invis + \[Sum]\+m\(\[LeftBracketingBar]\(f\^\[Prime]\)\_m - \(\ \@\[Lambda]\_m\/\(\[Lambda]\_m + \[Nu]\)\) \(g\^\[Prime]\)\_m\ \[RightBracketingBar]\^2\) \(\[Lambda]\_m + \[Nu]\)\/\[Lambda]\_m\)}], TraditionalForm]], "NumberedEquation", TextAlignment->AlignmentMarker, CounterIncrements->"NumberedEquation", CellTags->"Ed:LinkToBug"], Cell["\<\ This separates the contributions from the visible and the invisible \ subspaces.\ \>", "Text"] }, Closed]], Cell[CellGroupData[{ Cell["References", "Section"], Cell[TextData[{ "BERTERO, M., and PIKE, E. R., 1982, Resolution in diffraction limited \ imaging, a singular value analysis. Part 1: The case of coherent \ illumination. ", StyleBox["Optica Acta", FontSlant->"Italic"], ", ", StyleBox["29", FontWeight->"Bold"], ", 727-746." }], "Reference", CellTags->{"Ref:BerteroPike1982", "Ed:Change4"}], Cell[TextData[{ "COX, R. P., 1946, Probability, frequency and reasonable expectation. ", StyleBox["American Journal of Physics", FontSlant->"Italic"], ", ", StyleBox["17", FontWeight->"Bold"], ", 1-13." }], "Reference", CellTags->"Ref:Cox1946"], Cell[TextData[{ "JAYNES, E. T., 1968, Prior probabilities. ", StyleBox["I.E.E.E. Transactions on Systems Science and Cybernetics", FontSlant->"Italic"], ", ", StyleBox["4", FontWeight->"Bold"], ", 227-241." }], "Reference", CellTags->"Ref:Jaynes1968"], Cell[TextData[{ "JEFFREYS, H., 1939, ", StyleBox["Theory of Probability", FontSlant->"Italic"], " (Oxford: Oxford University Press)." }], "Reference", CellTags->"Ref:Jeffreys1939"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/ble/ble.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., 1985 a, Prior knowledge and object reconstruction using \ the best linear estimate technique. ", StyleBox["Optica Acta", FontSlant->"Italic"], ", ", StyleBox["32", FontWeight->"Bold"], ", 703-716." }], "Reference", CellTags->"Ref:Luttrell1985a"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/transinf/transinf.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., 1985 b, The use of transinformation in the design of data \ sampling schemes for inverse problems. ", StyleBox["Inverse Problems", FontSlant->"Italic"], ", ", StyleBox["1", FontWeight->"Bold"], ", 199-218." }], "Reference", CellTags->"Ref:Luttrell1985b"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/bs1_41/bs1_41.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., 1987 a, The relationship between super-resolution and \ phase imaging of SAR data. RSRE Technical Report 1987/BS1/41, Royal Signals \ and Radar Establishment, Malvern." }], "Reference", CellTags->"Ref:Luttrell1987a"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/4079/4079.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., 1987 b, The complex point spread function of the RSRE \ SAR. RSRE Memorandum 4079, Royal Signals and Radar Establishment, Malvern." }], "Reference", CellTags->"Ref:Luttrell1987b"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/invcross/invcross.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., 1989, The inverse cross section problem for complex data. \ ", StyleBox["Inverse Problems", FontSlant->"Italic"], ", ", StyleBox["5", FontWeight->"Bold"], ", 35-50." }], "Reference", CellTags->"Ref:Luttrell1989"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/priorsar/priorsar.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., and OLIVER, C. J., 1986, Prior knowledge in synthetic \ aperture radar processing. ", StyleBox["Journal of Physics D: Applied Phvsics", FontSlant->"Italic"], ", ", StyleBox["19", FontWeight->"Bold"], ", 333-356." }], "Reference", CellTags->"Ref:LuttrellOliver1986"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/prior/prior.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "LUTTRELL, S. P., and OLIVER, C. J., 1988, The r\[OHat]le of prior \ knowledge in coherent image processing. ", StyleBox["Philosophical Transactions of the Royal Society of London, A", FontSlant->"Italic"], ", ", StyleBox["324", FontWeight->"Bold"], ", 397-407." }], "Reference", CellTags->"Ref:LuttrellOliver1988"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/sr_alg/sr_alg.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "PRYDE, G. C., DELVES, L. M., and LUTTRELL, S. P., 1988 a, A \ super-resolution algorithm for SAR images. ", StyleBox["Inverse Problems", FontSlant->"Italic"], ", ", StyleBox["4", FontWeight->"Bold"], ", 681-703." }], "Reference", CellTags->"Ref:PrydeDelvesLuttrell1988a"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/conpar88/conpar88.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "PRYDE, G. C., DELVES, L. M., and LUTTRELL, S. P., 1988 b, A comparative \ study of the AMT DAP and of Transputer array architectures for the \ super-resolution of synthetic aperture radar images. ", StyleBox["Proceedings of CONPAR'88 held in Manchester, U.K., in 1988", FontSlant->"Italic"], " (Cambridge: Cambridge University Press), pp. 340-350." }], "Reference", CellTags->"Ref:PrydeDelvesLuttrell1988b"], Cell[TextData[{ Cell[TextData[{ " ", ButtonBox["OPEN", ButtonData:>{ URL[ "http://www.luttrell.org.uk/papers/ima89/ima89.nb"], None}, Active->True, ButtonStyle->"Hyperlink"], " " }]], "PRYDE, G. C., DELVES, L. M., and LUTTRELL, S. P., 1988 c, The performance \ of a parallel super-resolution algorithm for synthetic aperture radar images. \ ", StyleBox["Proceedings of the IMA Conference on Mathematics in Signal \ Processing held in Warwick, U.K., in 1988", FontSlant->"Italic"], " (Oxford: Oxford University Press), pp. 739-747." }], "Reference", CellTags->{"Ref:PrydeDelvesLuttrell1988c", "Ed:Change5"}], Cell[TextData[{ "SHANNON, C. E., and WEAVER, W., 1949, ", StyleBox["The Mathematical Theory of Communication", FontSlant->"Italic"], " (Chicago: University of Illinois Press)." }], "Reference", CellTags->"Ref:ShannonWeaver1949"], Cell[TextData[{ "SHORE, J. E., and JOHNSON, R. W., 1980, Axiomatic derivation of the \ principle of maximum entropy and the principle of minimum cross entropy. ", StyleBox["I.E.E.E. Transactions on Information Theory", FontSlant->"Italic"], ", ", StyleBox["26", FontWeight->"Bold"], ", 26-37; ", StyleBox["29", FontWeight->"Bold"], ", 942-943." }], "Reference", CellTags->"Ref:ShoreJohnson1980"], Cell[TextData[{ "SKILLING, J., 1989, ", StyleBox["Maximum Entropy and Bayesian Methods", FontSlant->"Italic"], " (Kluwer Academic Publishers)." }], "Reference", CellTags->"Ref:Skilling1989"] }, Closed]] }, Open ]] }, FrontEndVersion->"5.0 for Microsoft Windows", ScreenRectangle->{{0, 1280}, {0, 941}}, WindowSize->{665.375, 641}, WindowMargins->{{307.25, Automatic}, {Automatic, 50}}, Magnification->1, StyleDefinitions -> "Report.nb" ] (******************************************************************* Cached data follows. If you edit this Notebook file directly, not using Mathematica, you must remove the line containing CacheID at the top of the file. The cache data will then be recreated when you save this file from within Mathematica. *******************************************************************) (*CellTagsOutline CellTagsIndex->{ "Eq:ConsistencyConditions"->{ Cell[9228, 321, 496, 9, 70, "NumberedEquation", CellTags->{"Eq:ConsistencyConditions", "Ed:Change6"}]}, "Ed:Change6"->{ Cell[9228, 321, 496, 9, 70, "NumberedEquation", CellTags->{"Eq:ConsistencyConditions", "Ed:Change6"}]}, "Eq:BayesianCalculus"->{ Cell[10396, 354, 395, 8, 70, "NumberedEquation", CellTags->"Eq:BayesianCalculus"]}, "Eq:BayesTheorem"->{ Cell[11998, 400, 227, 4, 70, "NumberedEquation", CellTags->"Eq:BayesTheorem"]}, "Eq:ScatteringPDF"->{ Cell[14016, 470, 267, 5, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ScatteringPDF"]}, "Eq:ImagingModel"->{ Cell[19764, 631, 145, 3, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ImagingModel"]}, "Eq:ImagingModelPDF"->{ Cell[19963, 638, 665, 18, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ImagingModelPDF"]}, "Sect:InverseObjectFieldProblem"->{ Cell[21697, 698, 149, 6, 70, "Section", CellTags->"Sect:InverseObjectFieldProblem"]}, "Eq:JointPDF"->{ Cell[22190, 717, 265, 5, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:JointPDF"]}, "Eq:ConditionalPDF"->{ Cell[22458, 724, 694, 16, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF"]}, "Eq:ConditionalPDF2"->{ Cell[25691, 836, 803, 21, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF2"]}, "Eq:Covariance"->{ Cell[27923, 902, 965, 28, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Covariance"]}, "Eq:Frec"->{ Cell[29911, 969, 232, 5, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Frec"]}, "Sect:InformationChannels"->{ Cell[33241, 1068, 172, 9, 70, "Subsection", CellTags->"Sect:InformationChannels"]}, "Eq:ConditionalPDF3"->{ Cell[34330, 1106, 926, 23, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF3"]}, "Sect:MutualInformation"->{ Cell[42454, 1352, 168, 9, 70, "Subsection", CellTags->"Sect:MutualInformation"]}, "Eq:MutualInformation2"->{ Cell[46275, 1481, 212, 4, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation2"]}, "Eq:MutualInformation3"->{ Cell[46490, 1487, 223, 4, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation3"]}, "Eq:MutualInformation4"->{ Cell[47851, 1535, 708, 19, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation4"]}, "Eq:DataPDF"->{ Cell[50055, 1599, 772, 21, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:DataPDF"]}, "Sect:InverseCrossSectionProblem"->{ Cell[52989, 1693, 151, 6, 70, "Section", CellTags->"Sect:InverseCrossSectionProblem"]}, "Sect:FirstDerivative"->{ Cell[56605, 1809, 231, 11, 70, "Subsection", CellTags->"Sect:FirstDerivative"]}, "Eq:Likelihood"->{ Cell[57004, 1829, 586, 15, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Likelihood"]}, "Eq:Derivatives"->{ Cell[57672, 1848, 2205, 60, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Derivatives"]}, "Eq:LikelihoodDerivative"->{ Cell[60683, 1942, 1032, 27, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivative"]}, "Eq:Frec2"->{ Cell[61766, 1973, 1049, 28, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Frec2"]}, "Ed:Change1"->{ Cell[62818, 2003, 630, 19, 70, "Text", CellTags->"Ed:Change1"]}, "Eq:LikelihoodDerivative2"->{ Cell[65655, 2091, 398, 7, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivative2"]}, "Eq:PosteriorDerivative"->{ Cell[68336, 2169, 385, 6, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:PosteriorDerivative"]}, "Eq:LikelihoodDerivativeN"->{ Cell[71572, 2269, 2938, 68, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivativeN"]}, "Sect:Appendix"->{ Cell[79793, 2507, 96, 1, 70, "Section", CellTags->"Sect:Appendix"]}, "Eq:SVD"->{ Cell[80043, 2517, 511, 10, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:SVD"]}, "Ed:Change2"->{ Cell[80557, 2529, 1040, 36, 70, "Text", CellTags->"Ed:Change2"]}, "Ed:Change3"->{ Cell[83992, 2645, 308, 6, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Ed:Change3"]}, "Ed:LinkToBug"->{ Cell[85563, 2699, 852, 20, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Ed:LinkToBug"]}, "Ref:BerteroPike1982"->{ Cell[86591, 2731, 359, 11, 70, "Reference", CellTags->{"Ref:BerteroPike1982", "Ed:Change4"}]}, "Ed:Change4"->{ Cell[86591, 2731, 359, 11, 70, "Reference", CellTags->{"Ref:BerteroPike1982", "Ed:Change4"}]}, "Ref:Cox1946"->{ Cell[86953, 2744, 264, 9, 70, "Reference", CellTags->"Ref:Cox1946"]}, "Ref:Jaynes1968"->{ Cell[87220, 2755, 271, 9, 70, "Reference", CellTags->"Ref:Jaynes1968"]}, "Ref:Jeffreys1939"->{ Cell[87494, 2766, 192, 6, 70, "Reference", CellTags->"Ref:Jeffreys1939"]}, "Ref:Luttrell1985a"->{ Cell[87689, 2774, 505, 19, 70, "Reference", CellTags->"Ref:Luttrell1985a"]}, "Ref:Luttrell1985b"->{ Cell[88197, 2795, 531, 20, 70, "Reference", CellTags->"Ref:Luttrell1985b"]}, "Ref:Luttrell1987a"->{ Cell[88731, 2817, 466, 14, 70, "Reference", CellTags->"Ref:Luttrell1987a"]}, "Ref:Luttrell1987b"->{ Cell[89200, 2833, 425, 13, 70, "Reference", CellTags->"Ref:Luttrell1987b"]}, "Ref:Luttrell1989"->{ Cell[89628, 2848, 489, 20, 70, "Reference", CellTags->"Ref:Luttrell1989"]}, "Ref:LuttrellOliver1986"->{ Cell[90120, 2870, 542, 20, 70, "Reference", CellTags->"Ref:LuttrellOliver1986"]}, "Ref:LuttrellOliver1988"->{ Cell[90665, 2892, 561, 19, 70, "Reference", CellTags->"Ref:LuttrellOliver1988"]}, "Ref:PrydeDelvesLuttrell1988a"->{ Cell[91229, 2913, 520, 19, 70, "Reference", CellTags->"Ref:PrydeDelvesLuttrell1988a"]}, "Ref:PrydeDelvesLuttrell1988b"->{ Cell[91752, 2934, 659, 18, 70, "Reference", CellTags->"Ref:PrydeDelvesLuttrell1988b"]}, "Ref:PrydeDelvesLuttrell1988c"->{ Cell[92414, 2954, 660, 18, 70, "Reference", CellTags->{"Ref:PrydeDelvesLuttrell1988c", "Ed:Change5"}]}, "Ed:Change5"->{ Cell[92414, 2954, 660, 18, 70, "Reference", CellTags->{"Ref:PrydeDelvesLuttrell1988c", "Ed:Change5"}]}, "Ref:ShannonWeaver1949"->{ Cell[93077, 2974, 240, 6, 70, "Reference", CellTags->"Ref:ShannonWeaver1949"]}, "Ref:ShoreJohnson1980"->{ Cell[93320, 2982, 423, 13, 70, "Reference", CellTags->"Ref:ShoreJohnson1980"]}, "Ref:Skilling1989"->{ Cell[93746, 2997, 202, 6, 70, "Reference", CellTags->"Ref:Skilling1989"]} } *) (*CellTagsIndex CellTagsIndex->{ {"Eq:ConsistencyConditions", 94645, 3024}, {"Ed:Change6", 94777, 3027}, {"Eq:BayesianCalculus", 94918, 3030}, {"Eq:BayesTheorem", 95035, 3033}, {"Eq:ScatteringPDF", 95149, 3036}, {"Eq:ImagingModel", 95308, 3040}, {"Eq:ImagingModelPDF", 95469, 3044}, {"Sect:InverseObjectFieldProblem", 95646, 3048}, {"Eq:JointPDF", 95761, 3051}, {"Eq:ConditionalPDF", 95917, 3055}, {"Eq:ConditionalPDF2", 96081, 3059}, {"Eq:Covariance", 96241, 3063}, {"Eq:Frec", 96390, 3067}, {"Sect:InformationChannels", 96549, 3071}, {"Eq:ConditionalPDF3", 96669, 3074}, {"Sect:MutualInformation", 96839, 3078}, {"Eq:MutualInformation2", 96960, 3081}, {"Eq:MutualInformation3", 97131, 3085}, {"Eq:MutualInformation4", 97302, 3089}, {"Eq:DataPDF", 97463, 3093}, {"Sect:InverseCrossSectionProblem", 97634, 3097}, {"Sect:FirstDerivative", 97760, 3100}, {"Eq:Likelihood", 97872, 3103}, {"Eq:Derivatives", 98029, 3107}, {"Eq:LikelihoodDerivative", 98197, 3111}, {"Eq:Frec2", 98359, 3115}, {"Ed:Change1", 98508, 3119}, {"Eq:LikelihoodDerivative2", 98615, 3122}, {"Eq:PosteriorDerivative", 98790, 3126}, {"Eq:LikelihoodDerivativeN", 98965, 3130}, {"Sect:Appendix", 99133, 3134}, {"Eq:SVD", 99226, 3137}, {"Ed:Change2", 99372, 3141}, {"Ed:Change3", 99466, 3144}, {"Ed:LinkToBug", 99617, 3148}, {"Ref:BerteroPike1982", 99778, 3152}, {"Ed:Change4", 99901, 3155}, {"Ref:Cox1946", 100025, 3158}, {"Ref:Jaynes1968", 100127, 3161}, {"Ref:Jeffreys1939", 100234, 3164}, {"Ref:Luttrell1985a", 100344, 3167}, {"Ref:Luttrell1985b", 100456, 3170}, {"Ref:Luttrell1987a", 100568, 3173}, {"Ref:Luttrell1987b", 100680, 3176}, {"Ref:Luttrell1989", 100791, 3179}, {"Ref:LuttrellOliver1986", 100907, 3182}, {"Ref:LuttrellOliver1988", 101029, 3185}, {"Ref:PrydeDelvesLuttrell1988a", 101157, 3188}, {"Ref:PrydeDelvesLuttrell1988b", 101291, 3191}, {"Ref:PrydeDelvesLuttrell1988c", 101425, 3194}, {"Ed:Change5", 101557, 3197}, {"Ref:ShannonWeaver1949", 101700, 3200}, {"Ref:ShoreJohnson1980", 101818, 3203}, {"Ref:Skilling1989", 101932, 3206} } *) (*NotebookFileOutline Notebook[{ Cell[CellGroupData[{ Cell[1776, 53, 26, 0, 70, "Section 1"], Cell[CellGroupData[{ Cell[1827, 57, 39, 0, 70, "Subsection"], Cell[1869, 59, 62, 0, 70, "Text"], Cell[1934, 61, 71, 0, 70, "Text"], Cell[2008, 63, 114, 3, 70, "Text"], Cell[2125, 68, 246, 8, 70, "Text"], Cell[2374, 78, 422, 14, 70, "Text"], Cell[2799, 94, 322, 13, 70, "Text"], Cell[3124, 109, 336, 13, 70, "Text"], Cell[3463, 124, 230, 8, 70, "Text"], Cell[3696, 134, 303, 9, 70, "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[4048, 149, 93, 2, 70, "Title"], Cell[4144, 153, 137, 4, 70, "Author"], Cell[4284, 159, 124, 3, 70, "Text"], Cell[4411, 164, 79, 0, 70, "Text"], Cell[4493, 166, 376, 7, 70, "Abstract"], Cell[CellGroupData[{ Cell[4894, 177, 85, 5, 70, "Section 1"], Cell[4982, 184, 299, 5, 70, "Text"], Cell[5284, 191, 731, 17, 70, "Text"], Cell[6018, 210, 988, 31, 70, "Text"], Cell[7009, 243, 627, 19, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[7673, 267, 83, 5, 70, "Section"], Cell[CellGroupData[{ Cell[7781, 276, 150, 8, 70, "Subsection"], Cell[7934, 286, 551, 11, 70, "Text"], Cell[8488, 299, 737, 20, 70, "Text"], Cell[9228, 321, 496, 9, 70, "NumberedEquation", CellTags->{"Eq:ConsistencyConditions", "Ed:Change6"}], Cell[9727, 332, 666, 20, 70, "Text"], Cell[10396, 354, 395, 8, 70, "NumberedEquation", CellTags->"Eq:BayesianCalculus"], Cell[10794, 364, 511, 8, 70, "Text"], Cell[11308, 374, 687, 24, 70, "Text"], Cell[11998, 400, 227, 4, 70, "NumberedEquation", CellTags->"Eq:BayesTheorem"], Cell[12228, 406, 672, 24, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[12937, 435, 151, 8, 70, "Subsection"], Cell[13091, 445, 922, 23, 70, "Text"], Cell[14016, 470, 267, 5, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ScatteringPDF"], Cell[14286, 477, 1402, 36, 70, "Text"], Cell[15691, 515, 1008, 25, 70, "Text"], Cell[16702, 542, 328, 10, 70, "Text"], Cell[17033, 554, 1184, 30, 70, "Text"], Cell[18220, 586, 1068, 26, 70, "Text"], Cell[19291, 614, 470, 15, 70, "Text"], Cell[19764, 631, 145, 3, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ImagingModel"], Cell[19912, 636, 48, 0, 70, "Text"], Cell[19963, 638, 665, 18, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ImagingModelPDF"], Cell[20631, 658, 1017, 34, 70, "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[21697, 698, 149, 6, 70, "Section", CellTags->"Sect:InverseObjectFieldProblem"], Cell[21849, 706, 338, 9, 70, "Text"], Cell[22190, 717, 265, 5, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:JointPDF"], Cell[22458, 724, 694, 16, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF"], Cell[23155, 742, 504, 17, 70, "Text"], Cell[23662, 761, 1086, 31, 70, "Text"], Cell[CellGroupData[{ Cell[24773, 796, 207, 10, 70, "Subsection"], Cell[24983, 808, 705, 26, 70, "Text"], Cell[25691, 836, 803, 21, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF2"], Cell[26497, 859, 54, 0, 70, "Text"], Cell[26554, 861, 1043, 24, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[27600, 887, 320, 13, 70, "Text"], Cell[27923, 902, 965, 28, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Covariance"], Cell[28891, 932, 797, 27, 70, "Text"], Cell[29691, 961, 217, 6, 70, "Text"], Cell[29911, 969, 232, 5, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Frec"], Cell[30146, 976, 222, 6, 70, "Text"], Cell[30371, 984, 1518, 40, 70, "Text"], Cell[31892, 1026, 989, 25, 70, "Text"], Cell[32884, 1053, 209, 5, 70, "Text"], Cell[33096, 1060, 108, 3, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[33241, 1068, 172, 9, 70, "Subsection", CellTags->"Sect:InformationChannels"], Cell[33416, 1079, 911, 25, 70, "Text"], Cell[34330, 1106, 926, 23, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:ConditionalPDF3"], Cell[35259, 1131, 1292, 38, 70, "Text"], Cell[36554, 1171, 1298, 40, 70, "Text"], Cell[37855, 1213, 3073, 87, 70, "Text"], Cell[40931, 1302, 1486, 45, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[42454, 1352, 168, 9, 70, "Subsection", CellTags->"Sect:MutualInformation"], Cell[42625, 1363, 699, 12, 70, "Text"], Cell[43327, 1377, 180, 8, 70, "Text"], Cell[43510, 1387, 329, 9, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[43842, 1398, 398, 8, 70, "Text"], Cell[44243, 1408, 1324, 42, 70, "Text"], Cell[45570, 1452, 265, 9, 70, "Text"], Cell[45838, 1463, 296, 11, 70, "Text"], Cell[46137, 1476, 135, 3, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[46275, 1481, 212, 4, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation2"], Cell[46490, 1487, 223, 4, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation3"], Cell[46716, 1493, 709, 22, 70, "Text"], Cell[47428, 1517, 420, 16, 70, "Text"], Cell[47851, 1535, 708, 19, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:MutualInformation4"], Cell[48562, 1556, 1490, 41, 70, "Text"], Cell[50055, 1599, 772, 21, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:DataPDF"], Cell[50830, 1622, 662, 19, 70, "Text"], Cell[51495, 1643, 286, 9, 70, "Text"], Cell[51784, 1654, 198, 4, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[51985, 1660, 955, 27, 70, "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[52989, 1693, 151, 6, 70, "Section", CellTags->"Sect:InverseCrossSectionProblem"], Cell[53143, 1701, 999, 22, 70, "Text"], Cell[54145, 1725, 117, 3, 70, "Text"], Cell[54265, 1730, 547, 13, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[54815, 1745, 526, 21, 70, "Text"], Cell[55344, 1768, 1236, 37, 70, "Text"], Cell[CellGroupData[{ Cell[56605, 1809, 231, 11, 70, "Subsection", CellTags->"Sect:FirstDerivative"], Cell[56839, 1822, 162, 5, 70, "Text"], Cell[57004, 1829, 586, 15, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Likelihood"], Cell[57593, 1846, 76, 0, 70, "Text"], Cell[57672, 1848, 2205, 60, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Derivatives"], Cell[59880, 1910, 800, 30, 70, "Text"], Cell[60683, 1942, 1032, 27, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivative"], Cell[61718, 1971, 45, 0, 70, "Text"], Cell[61766, 1973, 1049, 28, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:Frec2"], Cell[62818, 2003, 630, 19, 70, "Text", CellTags->"Ed:Change1"], Cell[63451, 2024, 598, 15, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[64052, 2041, 1358, 38, 70, "Text"], Cell[65413, 2081, 239, 8, 70, "Text"], Cell[65655, 2091, 398, 7, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivative2"], Cell[66056, 2100, 318, 7, 70, "Text"], Cell[66377, 2109, 1956, 58, 70, "Text"], Cell[68336, 2169, 385, 6, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:PosteriorDerivative"], Cell[68724, 2177, 1285, 38, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[70046, 2220, 197, 10, 70, "Subsection"], Cell[70246, 2232, 730, 18, 70, "Text"], Cell[70979, 2252, 422, 7, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[71404, 2261, 165, 6, 70, "Text"], Cell[71572, 2269, 2938, 68, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:LikelihoodDerivativeN"], Cell[74513, 2339, 614, 22, 70, "Text"] }, Closed]] }, Closed]], Cell[CellGroupData[{ Cell[75176, 2367, 82, 5, 70, "Section"], Cell[75261, 2374, 904, 16, 70, "Text"], Cell[76168, 2392, 2642, 87, 70, "Text"], Cell[78813, 2481, 943, 21, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[79793, 2507, 96, 1, 70, "Section", CellTags->"Sect:Appendix"], Cell[79892, 2510, 148, 5, 70, "Text"], Cell[80043, 2517, 511, 10, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Eq:SVD"], Cell[80557, 2529, 1040, 36, 70, "Text", CellTags->"Ed:Change2"], Cell[81600, 2567, 195, 9, 70, "Text"], Cell[81798, 2578, 360, 8, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[82161, 2588, 171, 8, 70, "Text"], Cell[82335, 2598, 668, 11, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[83006, 2611, 983, 32, 70, "Text"], Cell[83992, 2645, 308, 6, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Ed:Change3"], Cell[84303, 2653, 388, 15, 70, "Text"], Cell[84694, 2670, 498, 13, 70, "NumberedEquation", CounterIncrements->"NumberedEquation"], Cell[85195, 2685, 365, 12, 70, "Text"], Cell[85563, 2699, 852, 20, 70, "NumberedEquation", CounterIncrements->"NumberedEquation", CellTags->"Ed:LinkToBug"], Cell[86418, 2721, 104, 3, 70, "Text"] }, Closed]], Cell[CellGroupData[{ Cell[86559, 2729, 29, 0, 70, "Section"], Cell[86591, 2731, 359, 11, 70, "Reference", CellTags->{"Ref:BerteroPike1982", "Ed:Change4"}], Cell[86953, 2744, 264, 9, 70, "Reference", CellTags->"Ref:Cox1946"], Cell[87220, 2755, 271, 9, 70, "Reference", CellTags->"Ref:Jaynes1968"], Cell[87494, 2766, 192, 6, 70, "Reference", CellTags->"Ref:Jeffreys1939"], Cell[87689, 2774, 505, 19, 70, "Reference", CellTags->"Ref:Luttrell1985a"], Cell[88197, 2795, 531, 20, 70, "Reference", CellTags->"Ref:Luttrell1985b"], Cell[88731, 2817, 466, 14, 70, "Reference", CellTags->"Ref:Luttrell1987a"], Cell[89200, 2833, 425, 13, 70, "Reference", CellTags->"Ref:Luttrell1987b"], Cell[89628, 2848, 489, 20, 70, "Reference", CellTags->"Ref:Luttrell1989"], Cell[90120, 2870, 542, 20, 70, "Reference", CellTags->"Ref:LuttrellOliver1986"], Cell[90665, 2892, 561, 19, 70, "Reference", CellTags->"Ref:LuttrellOliver1988"], Cell[91229, 2913, 520, 19, 70, "Reference", CellTags->"Ref:PrydeDelvesLuttrell1988a"], Cell[91752, 2934, 659, 18, 70, "Reference", CellTags->"Ref:PrydeDelvesLuttrell1988b"], Cell[92414, 2954, 660, 18, 70, "Reference", CellTags->{"Ref:PrydeDelvesLuttrell1988c", "Ed:Change5"}], Cell[93077, 2974, 240, 6, 70, "Reference", CellTags->"Ref:ShannonWeaver1949"], Cell[93320, 2982, 423, 13, 70, "Reference", CellTags->"Ref:ShoreJohnson1980"], Cell[93746, 2997, 202, 6, 70, "Reference", CellTags->"Ref:Skilling1989"] }, Closed]] }, Open ]] } ] *) (******************************************************************* End of Mathematica Notebook file. *******************************************************************)