A prior chosen from noisy pictures keeps most of its gain, and a wrong one loses its alibi

A prior the pictures can refute is the one worth having took the problem four cameras and a field they cannot see had posed — a few parallel-ray views of a flow with no symmetry leave most of the field unseen — and filled the unseen part with an assumption. Four were tried on a 16 × 16 field: none at all, which sets the unseen part to zero; the smoothest field that meets the rays; a field that is never negative; and a sum of 64 Gaussian puffs. With exact rays the outcome was a ranking and a moral. Non-negativity saved three views of ten, smoothness one, the dictionary none unless it was exactly right; and the priors that only choose among fields meeting the data could not be caught when wrong, while the ones that exclude fields the data allow missed their own rays and so gave themselves away.

Its rays were exact, and it said what that hid. A real camera adds noise, and with noise every prior needs a weight: how far to trust the rays against the assumption. That essay’s noisy cases chose each weight by looking at the true field and taking the one with the least error, which is the most a prior could give and is not something an experiment can do. The question left was what each prior buys when its weight is chosen from the data alone, and whether a constraint’s refutation survives once its miss is comparable with the noise.

The ranking survives, and the refutation does not, quite. Chosen from the data, the weights cost the two unassuming priors almost nothing and non-negativity two or three views. But a wrong dictionary that missed its exact rays by four per cent is invisible at noise of six or seven, and above that it is not merely uncaught: it is about as accurate as the priors that were right.

Choosing a weight without the answer #

Each prior has a dial. The minimum-norm and smoothest-field reconstructions minimise the misfit to the rays plus a weight times a penalty — the field’s size, or its curvature — and as the weight falls the reconstruction meets the rays more and more closely. The dictionary has a ridge on its 64 amplitudes. Non-negativity has no weight at all; it is computed by a projected gradient method that starts from zero and creeps towards the best non-negative fit, and the number of steps it is allowed is its dial, because an iterative method stopped early has not yet fitted the fine detail, and fine detail is where the noise lives.

All four dials share one fact. Turned all the way towards the data, the reconstruction fits the noise as well as the field, and a field that reproduces the noise is wrong. Turned all the way towards the assumption, it ignores the data. Somewhere between is the setting that fits the rays as well as they deserve and no better, and if the noise level is known — and for a camera it is usually measured — that setting has a name. Morozov’s discrepancy principle takes the most heavily weighted reconstruction whose misfit to the rays has fallen to the noise level: fit the data to within its own error, and let the prior decide everything finer.

The rays here carry Gaussian noise whose root-mean-square is a stated share of the root-mean-square ray, one, three or ten per cent, drawn from a seeded random sequence; a check on five seeds finds the drawn level within one and a half per cent of the asked one. Each prior’s path is computed over forty settings of its dial, the discrepancy principle picks one, and the weight with the least error against the truth — the oracle the earlier essay used — is kept beside it for comparison.

Fitted to the noise, a reconstruction is ruined #

The first figure shows one dial turned through its range. The smoothest-field prior reconstructs a field of two puffs from eight views with three per cent ray noise. At heavy weight the reconstruction is a smooth blur that misses the rays badly and the field nearly as badly. As the weight relaxes the misfit falls and so does the error, until the misfit reaches the noise level. There the discrepancy principle stops, and the error is 14 per cent. The best weight, which only the true field could identify, is a little lighter and gives 12.

Past it the two curves part company. The misfit keeps falling, to a third of the noise level, because the reconstruction is now explaining the noise; and the error rises without limit, to eighty-six times the field itself at the lightest weight, because the null space the views leave — the patterns that add up to nothing along every ray — is no longer held down by anything, and the noise has been projected into it. An unregularised reconstruction of noisy rays is not a slightly worse reconstruction. It is no reconstruction at all, and the weight is the whole of the method.

What a weight chosen from the data costs #

The second figure gives the answer the earlier essay asked for, in views: how many each prior needs to bring a field of two puffs within fifteen per cent of the truth, against the noise. With its weight chosen by the discrepancy principle, no prior needs 8 views at one per cent noise and at three, and more than 24 at ten, where it never gets there at all. The smoothest field needs 8, 8 and 13. Non-negativity needs 4, 6 and 10. The dictionary needs 9, 9 and 15.

The oracle, which knows the answer, needs 8, 8 and more than 24 with no prior; 7, 8 and 11 with smoothness; 4, 4 and 7 with non-negativity; 8, 8 and 11 with the dictionary. For the two priors that only choose among fields meeting the data, the data-chosen weight costs a view at most, until the noise is large. For those the discrepancy principle is close to the oracle because the error and the misfit move together along the path until the misfit reaches the noise; the principle stops at almost the right place.

Non-negativity pays more: two views at three per cent noise and three at ten. The reason is in how its dial works. Early stopping is a cruder regulariser than a penalty — the iterations add detail in no particular order — and stopping at the first iteration whose misfit reaches the noise leaves the reconstruction short of the iterations that would have sharpened the puffs without fitting much noise. Still, the ranking stands: at every noise level here non-negativity needs the fewest views, and at ten per cent noise it reaches fifteen per cent with ten views where assuming nothing never does.

Noise puts a floor under every prior #

The third figure follows the error with views at a dialled noise level. Without noise every curve fell to zero once the views were enough to remove the null space, fourteen for this grid. With noise they level off. At one per cent noise and twelve views the errors are 4.6 per cent with no prior, 4.1 smooth, 3.3 non-negative; at twenty-four, 3.1, 2.2 and 2.3. At three per cent the same six numbers are 8.4, 7.3, 6.2, 6.9, 5.2 and 4.9. At ten per cent, 19, 16, 14 and then 16, 13 and 10.

Doubling the views from twelve to twenty-four takes between a sixth and a half off the error: more rays average the noise down, as more frames average down the noise in a run of frames with noise in every one, but slowly, and the floor is set by the noise rather than by the null space once the views are many. The dictionary sits apart: 11 per cent at one and three per cent noise whatever the views. That is the first sign of something the figure does not yet explain, because a field of two puffs is nearly a sum of dictionary puffs, and a prior that is nearly right should not stall.

A constraint’s miss has to stand above the noise #

The explanation is the refutation the earlier essay found, seen against the noise. The fourth figure takes each constraint applied to a field it cannot hold and asks for the smallest misfit it can reach on exact rays, at any weight. Non-negativity on a field with a trough cannot get closer than 77 per cent: it has to put something positive where the trough is and cannot. A dictionary of puffs on the earlier essay’s field with a streak cannot get closer than 4.2 per cent at eight views; on the field of two puffs, whose widths are not the dictionary’s, 3.8 per cent.

That last number is the stall. The puffs field is nearly but not exactly in the dictionary, so the dictionary’s best fit misses its rays by 3.8 per cent; at one and three per cent noise the discrepancy principle cannot reach the noise level, the dictionary is refused by the data, and its reconstruction is its best possible fit, 11 per cent wrong. The data are saying the prior is wrong, and they are right.

At ten per cent noise they say nothing. A constraint is caught only while its least misfit stands above the noise, and the line it has to clear is not quite the noise itself: the dictionary’s 64 amplitudes can absorb part of any noise, so the noisy rays’ best fit misses by less than the noise plus the floor would suggest. On the field with a streak the data stop refusing the dictionary at a noise of 6.2 per cent from eight views and 8.1 from twelve, one and a half to one and three-quarter times the floor. For the field of two puffs the crossover is at 5.7 and 6.8. Non-negativity’s 77 per cent is never approached by any noise a measurement would keep.

Above the floor a wrong prior has no tell #

The fifth figure shows what crossing that line means. The field has a streak no sum of puffs can represent; the dictionary is wrong about it in kind. At low noise the data refuse it and its reconstruction is its best fit, the field with the streak smeared into puffs. Above eight per cent noise at twelve views it fits the noisy rays to within their noise, exactly as the other priors do, and the discrepancy principle cannot tell it from them. At ten per cent noise its error is 20 per cent and the smoothest field’s 19.

The wrong prior has not become right; the noise has made everyone equally wrong. That is the shape of the refutation’s failure, and it is worth stating precisely because it is easy to read the other way. The earlier essay’s moral was that a prior the data can refute is worth having, because when it is wrong the data say so. With noise the moral needs a number attached: the data say so only for errors in the prior that are larger than about one and a half times the noise, measured as the misfit they cause. A dictionary nearly right — puffs of slightly different widths — and a dictionary wrong in kind — a streak where it has only puffs — both fall under that line at the noise levels of a careful schlieren or background-oriented measurement, which are a few per cent.

What remains is the hard constraint. Non-negativity on a signed field misses by 77 per cent, and a density deviation measured against ambient is exactly such a field; nobody would apply that prior to it twice. The constraints worth having are the ones whose violations are gross, and a constraint subtle enough to be nearly right is subtle enough to be hidden.

Where a ray’s noise comes from #

A ray here is a line integral of the field, and no optical instrument measures one directly. A schlieren system or a background-oriented camera measures how far light is bent, which is the integral of the field’s gradient across the ray — an instrument that takes a derivative — and the ray integral is recovered by integrating that deflection along the image. The integration is where most of the noise enters. A deflection is measured to a fraction of a pixel by correlating a displaced pattern, the same operation that sets the random error in every particle-image vector and that the window every vector is averaged over traced to the size of the correlation window; integrated across an image a few hundred pixels wide, independent pixel errors accumulate as a random walk, so the integrated ray is noisier at the far side of the image than at the near.

That has two consequences for the arithmetic above. The noise level is not one number but a profile across each view, and the discrepancy principle then wants the misfit weighted by each ray’s own variance — a change that leaves its logic alone and makes its δ a vector. And the noise is correlated along the image, rays near each other sharing the integration’s early steps, which is the structure listed below as left out. In practice a few per cent of the root-mean-square ray is a good measurement, which is the middle of the range drawn here, and the place where the dictionary’s refutation is lost.

What a photograph of the flow could add is not more rays but an independent check on the prior: a single seeded or shadow picture of where the flow is marked — the kind of evidence what a photograph of a flow shows weighs — constrains the support of the field without any ray at all, and a support constraint is a hard one, refused by gross misfits, like non-negativity.

What the experimenter controls #

Three things move the line. More views help twice: they lower every error floor, and they raise the noise a wrong dictionary can hide behind, from 6.2 to 8.1 per cent between eight and twelve views on the streak field, because more rays leave the 64 amplitudes less room to absorb the noise. A better measurement of the noise level helps: the discrepancy principle can only be as good as its δ, and a δ underestimated by half drives every unconstrained prior into fitting the noise, while one overestimated leaves them all blurred. And a prior with fewer free amplitudes is harder to hide: a dictionary of sixteen puffs would absorb a quarter as much noise, so the data would refuse it up to a noise much closer to twice its floor — at the cost of a higher floor, since sixteen puffs are nearly right about fewer fields.

What was checked #

With no noise, the smoothest-field path ends, at its lightest weight, meeting its rays to four parts in a billion — the earlier essay’s clean reconstruction. The drawn noise, asked for five per cent, gives 5.07 per cent averaged over five seeds. For no prior at twelve views and three per cent noise the discrepancy principle and the oracle choose the same setting and the same error, 8.40 per cent, which is the case where the principle is known to be near-optimal. The dictionary on the streak field is refused at three per cent noise and accepted at twelve; non-negativity on the trough field is refused at three and at twenty. The reconstruction refuses a negative noise level and an unknown prior.

What the model leaves out #

One noise draw. Every number is from a single seeded draw of the noise. The views counts are integers and move by one between draws near their thresholds; the refutation thresholds move by a few tenths of a per cent.

Known noise. The discrepancy principle needs the noise level. Without it, methods such as the L-curve or generalised cross-validation choose the weight from the shape of the path alone, and they are generally less reliable at small view counts than a measured δ.

Gaussian, independent rays. Real ray noise has structure — neighbouring rays share pixels of the same camera — and correlated noise lies partly along the field’s own patterns, which makes it both harder to average and easier for a dictionary to absorb.

A grid of 256 cells. The priors’ gains in views scale with the resolution, as one view is enough, and the axis pays for it found for the symmetric case; the refutation’s arithmetic, a misfit floor against a noise level, does not.

Morozov, and the tomographers who use him #

The discrepancy principle is Morozov’s, from 1966, and its properties for Tikhonov regularisation — that the reconstruction converges as the noise falls, at a rate set by the field’s smoothness — are standard. Early stopping as regularisation for iterative reconstruction, with the stopping index chosen the same way, goes back to the Landweber iteration and is how many practical tomographic codes, including those for tomographic particle-image velocimetry, decide when to stop. What this calculation adds is the comparison within one problem: what each prior keeps of its noise-free advantage when the weight is chosen honestly, and the measured level of noise at which a constraint’s refutation is lost. Related reading on what a reconstruction is entitled to say about the parts of a flow no measurement determines is in the inside a flow does not decide.

Still open: a prior that adapts #

The three priors here are fixed in advance. A modern reconstruction learns its prior — a dictionary fitted to earlier snapshots of the same flow, or a sparsity penalty that chooses its own atoms — and so has many more free amplitudes than sixty-four. On the arithmetic above that makes it easier to fit any noisy rays and harder to refute. The next calculation would learn a dictionary from a sequence of reconstructions of a flow that slowly changes, apply it to a later frame in which a new structure has appeared, and ask whether the data can still refuse the learned prior at the noise level the earlier frames were measured at, or whether a prior trained on the past quietly paints the past into the present.