Two surprises and one event

Expectation is a curve, not a list gave this ladder a second surprise. The first four rungs answer which chord a listener expects, and the fifth answers when — a chord change is more likely on a strong beat than a weak one, so its arrival carries an information cost of its own.

Its last paragraph names what that leaves undone:

There are two surprises available at every chord change and a listener meets one event, so the honest quantity is the total — and adding them requires knowing whether they are independent.

Two surprises are two negative log probabilities. If the events are independent the total is the sum. If one implies the other, the total is the larger. Everything in between is a stated amount of shared information, and which it is depends on whether unusual chords tend to arrive at unusual moments — a fact about a repertoire that nobody has counted.

So it is swept.

The two are not the same size #

Before adding them it is worth seeing them side by side, because they are not comparable quantities in the way “two surprises” suggests.

Across the progression the identity surprise runs from 1.85 to 2.85 bits and the timing surprise from 2.02 to 4.52. The timing surprise is the larger at every event, and by a factor of two at some of them.

That is a consequence of the fifth rung’s own arithmetic rather than a finding here. A chord change on a strong beat is likely and cheap; one on a weak beat is unlikely and expensive, and the metrical hierarchy of a bar of four spans a factor of about six in hazard. The identity surprise spans much less, because the root-motion distribution is flatter than the metrical one: a fifth down is common, a third up is not, and the ratio between them is under three.

So most of what a listener is surprised by at a chord change is when it arrived, not what it was — on this model, at this harmonic rate, in this metre.

What the correlation is worth #

Adding them under a family that runs from independence to identity gives a mean total of 5.78 bits at one end and 3.79 at the other. A factor of 1.53.

That is the size of the missing number. It is not small — half again on every reported quantity — and it means any single figure this rung produced would be quoting a number with a fifty per cent bracket on it.

The ordering is a different story. Ranking the seven chord changes of the passage by total surprise gives exactly the same order at every correlation swept up to 0.8, and the most surprising event is the same one all the way to 1.0. Only at perfect dependence — where the two surprises are literally the same event — do three of the seven swap places, and even then the top of the list does not move.

The corpus would change what this rung reports and not what it concludes.

Why a factor of one and a half is the right thing to report #

There is a temptation, faced with a missing parameter, to pick a plausible value and get on with it. Independence is the usual pick, because it is what a model does when nobody has thought about the question, and it is the worst end of this family — it gives the largest total at every event.

Picking it silently would have been an overstatement of every number this rung produces, by up to fifty-three per cent, with no flag on it. That is the failure mode the sweep exists to prevent, and it is the same one the key ladder’s own missing number has: a parameter chosen by default is a parameter nobody has read.

So the reportable form of the total is a range, and the range is stated with its two ends named rather than as an error bar. At the independent end a chord change in this passage costs a listener 5.78 bits on average; at the dependent end, 3.79. A corpus would pick a point in between and would not move anything outside it.

Which is the useful shape for a debt to have #

This anchor, the key ladder and the progression ladder have each recorded wanting the same corpus of harmonic analyses, three times independently, and this rung’s version of that debt was the correlation between the two surprises.

What the sweep says is that the debt is real and is not blocking. The quantity that needs the corpus is a scale factor, and the arguments this ladder makes are about which moments in a passage carry the surprise. Those survive.

That is worth separating from the other two ladders’ versions of the same debt, because theirs do not have this property. The progression ladder’s is a coefficient in a model with two collinear predictors, and its whole finding is that the coefficient cannot be got from a single passage at all. This one is a correlation whose effect on the ranking is nil.

A shared debt is not one debt. Three ladders wanting one corpus turns out to be three ladders wanting different things from it, and only one of them is stuck. That is worth recording where the debts are, because a corpus that would unblock one ladder and refine two is a different proposition from one that would unblock three.

The two surprises are also two ladders #

There is a structural point in the sizes worth recording, because it explains why this rung was late.

The identity surprise belongs to the progression ladder, which counts root motions. The timing surprise belongs to the metre ladder, which counts where changes fall. This anchor sits between them and has been drawing on both since its second rung, and the two were built four phases apart with no reason to be commensurable.

That they come out in the same units is not luck — both are negative log probabilities and were built that way deliberately — but that they come out the same size is. A root-motion distribution and a metrical hierarchy have nothing to do with each other, and the spread of the second happens to be about twice the spread of the first.

If it had been a factor of a hundred there would be no question here, because the total would be one of them and the correlation would be irrelevant. It is a factor of two, which is exactly the regime where the missing number does the most work — and it is also why the answer comes out as a factor of 1.53 rather than 2.

What would make them correlated, and it is a real mechanism #

The sweep is neutral about the sign of the correlation and the repertoire is not.

There is a good reason to expect a positive one. A composer writing a surprising chord tends to place it where it will be noticed, which is a metrically strong position — so the unusual chord and the unusual moment would tend to avoid each other, and the two surprises would be negatively correlated in the sense that matters.

There is an equally good reason to expect the opposite. A syncopated arrival is itself a marked gesture, and the passages that use one often use the other at the same time, which is a positive correlation.

Both are plausible, they point opposite ways, and this collection cannot decide between them. The honest state is that the sign is unknown as well as the size, which is a stronger statement of the debt than the fifth rung made — and it is also why the family swept here runs only from independence to identity rather than to anticorrelation, which would need a different construction and has no natural upper end.

The harmonic rate is the variable that decides which surprise dominates #

The claim above — that most of what a listener meets at a chord change is the timing — is not a fact about music. It is a fact about music at one chord a bar.

The timing surprise is a hazard scaled to the harmonic rate, so as the rate rises every beat becomes a likely place for a change and the surprise of arriving at one falls. At four chords a bar the timing surprise nearly vanishes and the identity surprise is untouched, because a root motion is as unlikely as it ever was.

So the balance between the two is set by the harmonic rhythm, and a slow-moving passage is one in which when carries most of the information while a fast-moving one is all what. That is a difference between a chorale and a fast toccata that this ladder can now state in bits.

It also says where the correlation would matter most. The family’s two ends differ most when the two surprises are similar in size, because the total then falls from 2a to a; when one dominates, the sum and the maximum are nearly the same number. The correlation is worth least exactly where one surprise is much larger than the other, which on this progression is at every event — and is why the factor of 1.53 is not larger.

What the ordering result is and is not #

The invariance of the ordering is the reportable result and it is easy to overclaim.

It says that ranking the events of this progression by total surprise gives the same answer for any correlation short of perfect dependence. It does not say that the ranking is right, or that a listener has one, or that the same would hold on a passage built to break it.

What it does do is tell the anchor which of its statements are safe. Every claim of the form this moment carries the most of a passage’s surprise survives the missing number. Every claim of the form this passage carries N bits does not, and should be reported with the range.

Which computation produced the numbers #

The identity surprise is chordSurprise, unchanged from the third rung: the probability of a root motion under the published frequencies, in bits.

The timing surprise is the fifth rung’s, unchanged: the metrical hazard at the beat where the change arrives, scaled so that the expected number of changes in a bar comes out at the harmonic rate, in bits.

The family is a + b − ρ·min(a, b). At ρ = 0 it is the sum, which is what independence gives; at ρ = 1 it is the larger of the two, which is what perfect dependence gives. Nothing about the interpolation between them is derived — it is the simplest curve with the right two ends.

The passage is a seven-change progression at one chord a bar in four-four, with the changes on downbeats and one syncopation.

What a total is for #

It is worth asking what the sum would be used for, because a quantity nobody can act on is not worth a corpus.

The answer this ladder has is that a total is what a listener meets. Every rung before the fifth reported a surprise per chord and treated the metrical position as a separate matter; the fifth showed it is not separate, and this one says how the two combine. What comes out is a single number per event, which is the right shape for the thing a listener has: one moment, one degree of astonishment.

That number is then the input to the questions this anchor exists for. Which chord change in a phrase is the one a listener remembers; whether a passage’s surprise is concentrated or spread; whether two settings of the same words differ in where their surprise falls. All of those are orderings, and all of them survive.

The total’s absolute size is only wanted for comparing passages, and comparing passages in bits is a thing this collection has been careful about since the first rung — the units are model units, and two passages differ in more than their surprise.

Where the model stops #

A correlation is not a copula. Two surprises can share information in ways a single parameter cannot express, and the family here fixes both ends correctly and everything between by fiat.

The hazard is a shape, not a measurement. The metrical weights come from a published hierarchy and the harmonic rate is asserted; the ladder’s own rate figure sweeps the second and not the first.

Both surprises are model outputs. Neither is a measurement of a listener, and the whole apparatus assumes a listener whose expectations match the published frequencies of a repertoire they may not know.

And seven events is a passage. The ordering-invariance result is about this progression; a passage engineered to put a very surprising chord at a very unsurprising moment would be one where the correlation could reorder things, and nobody has built one.

What the picture cannot show #

It cannot show what a listener does with two surprises. Adding them is a modelling convenience; a listener might attend to one and not the other, or might have separate responses to each.

Nor can it show the melody or the metre being ambiguous. The timing surprise assumes the listener knows where the downbeat is, which is a whole ladder’s worth of question on this site.

It cannot show a repertoire. Every probability here is a published count of common-practice European harmony, applied to a progression that is not from it, which is this ladder’s standing compromise and is stated wherever both are used.

It cannot show a listener who knows the piece. Every probability here is a first hearing’s, and a second hearing has no identity surprise at all — which the repetition ladder is about and which would remove one of the two terms entirely.

And it cannot show anticipation. A listener does not only receive a surprise; they carry an expectation forward, and the fifth rung’s survival term is the part of that this ladder has — with a defect it published rather than hid.

Whose music, and when #

The root-motion frequencies are counts over common-practice European tonal music; the metrical hierarchy is the same tradition’s. The progression is generic.

There is no historical claim available here, and it is worth saying so. The one that would be available with a corpus is the interesting one: whether the correlation between the two surprises differs between repertoires — whether, say, a nineteenth-century repertoire places its unusual chords more carefully than an eighteenth-century one. That is a difference two numbers would settle and it is exactly the shape of question this ladder was opened to ask.

Where this ladder goes next #

Six rungs. Counting produced the hierarchy; a chord did not come and the moment was described; the moment acquired a number; the number acquired an uncertainty; the number acquired a companion, for when rather than for which; and now the two added, with the thing that joins them swept because nobody has it.

What is owed after this is the third surprise. Both quantities here are about a chord change, and a listener meets a great deal of music in which nothing changes: the fifth rung already computes what it costs to be surprised by a chord that does not arrive, and that cost is paid at every beat rather than at every change. Summed over a bar it is a running quantity of the same kind as the two here, in the same units, and it is much larger than either because there are more beats than changes. Whether a listener’s total expectation is dominated by the events or by the non-events is a question with an arithmetic rather than an opinion behind it, and this ladder has both halves.