Expectation is a curve, not a list

Four rungs of this ladder produce a list. Counting produced the hierarchy gives a number per scale degree, surprise is a number gives one per chord change, and a chord, given a key and a predecessor gives a better one. Every quantity in the ladder is attached to an event.

Every published account of musical expectation is drawn as a curve. Something rises as a bar proceeds and collapses when the chord changes, and the shape of that rise is the thing the accounts are about. Its last rung named the gap:

The progression ladder has a model of how often the chord changes and the metre ladder has a model of where the strong beats are, so the ingredients for an expectation that rises and falls within a bar are here.

They are, and joining them produces a term this ladder has never had.

A hazard is the shape of the join #

The quantity that makes the two ingredients fit together is a hazard: given that the chord has not changed yet, how likely is it to change on this beat.

That is the right shape for two reasons. It is what a listener has — the question is always asked from inside the bar, with what has happened so far known and what has not unknown — and it is the only form in which the two ingredients are commensurable.

The metre supplies the shape. The bar above the bar is this collection’s account of the metrical hierarchy: beats are not equal, and a bar of four has a strongest position, a second-strongest at the halfway point, two weaker ones at the other beats and much weaker ones between. Those weights are the profile.

The harmonic rhythm supplies the scale. How often the chord changes measures the rate at which harmony moves, in changes per bar, and that number fixes how much total probability the profile has to distribute.

Scaling the metrical weights so that the expected number of changes in a bar comes out at the measured rate turns a shape into a set of probabilities. That is the whole construction, and it is one line.

The two things it separates #

The construction has a property worth stating because it is what makes the join informative rather than merely possible.

The ratio between the strongest and weakest beats is fixed by the metre alone, at 6.67 for the weights this collection uses, and it does not depend on the harmonic rhythm at all. Doubling the rate at which chords change doubles every hazard and leaves their ratios exactly where they were.

The absolute level is fixed by the harmonic rhythm alone.

So the two ingredients contribute orthogonally: one decides the shape and one the height, and they can be varied independently right up to the point where the strongest beat’s hazard approaches one and the profile has to saturate. That happens at about four changes a bar, which is a chord on every beat — where there is nothing left for a hierarchy to say.

That separation is the reason it is worth computing at all. A model in which the two ingredients interacted would be a model in which the metre’s contribution could not be read off, and the whole appeal of this construction is that it can.

A second surprise, which the ladder has never had #

With a hazard in hand there are two quantities available at every beat, and they are different things.

The identity surprise is what the ladder already computes: given that a change has arrived, how unlikely was this chord. That is the conditional surprise — root-motion improbability times profile fit — and it has nothing to do with when.

The timing surprise is new: how unlikely was a change here. It is −log₂ of the hazard when a change arrives, and −log₂ of one minus the hazard when the bar goes by without one.

A listener meets the sum, and the sum behaves in a way neither term does alone.

Take a two-bar phrase with chords changing on the downbeats. The timing bill is 3.57 bits. Take the same two changes placed off the beat and it is 9.05 bits. A flat expectation — a listener with no metre at all — charges 6.00 bits either way, because a flat hazard does not care where anything falls.

So the metre makes an on-beat change 2.43 bits cheaper than it would otherwise be and an off-beat change 3.05 bits dearer. The asymmetry runs the way the flat control cannot: a metre gives back less on the beat than it takes off it, which is the arithmetic behind a thing every theory of rhythm asserts, that a syncopation is expensive.

And the weights are asserted, so the ratio is swept #

Everything above follows from eight numbers that were written down rather than measured, and the honest thing is to try others. Two obvious alternatives are structural rather than stipulated: weight each position by how many metrical levels it belongs to (bar, half-bar, beat, quaver), and weight it by two to the power of that count, which is the same hierarchy read multiplicatively.

The direction is robust and the magnitude is not, which is what the caveat below predicts and is worth having as numbers. Every hierarchy makes the on-beat change cheaper and the off-beat change dearer, by construction, since all three put the downbeat highest. The saving runs from 2.19 to 3.36 bits and the charge from 1.81 to 3.05 — a spread of about 1.2 bits on each, which is a fifty per cent uncertainty on both.

One thing does not survive the sweep at all. The essay’s hierarchy charges more than it saves, 3.05 against 2.43; the level-doubling hierarchy saves more than it charges, 3.36 against 2.64; the level-count one charges less than it saves too. Whether a metre is net generous or net punitive is a property of the weights and not of metre, and any argument resting on that sign is resting on a number nobody measured.

What that says about syncopation #

Syncopation is a number about the metre measures the same phenomenon on the rhythm ladder, by counting notes that fall on weak positions and rests that fall on strong ones. That is a good measure and it is about onsets.

The timing surprise here is about harmonic events, and it is a different quantity with the same shape. A passage whose notes are all on the beat and whose harmonies change off it is not syncopated by the rhythm ladder’s measure and is expensive by this one — and that combination is a real and common musical device, from the anticipated cadential arrival to the harmony that changes on the last quaver of the bar.

So the two measures are complementary rather than competing, and joining them is a natural thing to want. What stops it here is that they are in different units: the rhythm ladder’s syncopation is a weighted count, and this is bits. Converting between them needs the very exchange rate this collection keeps avoiding.

That is the figure the essay’s title is about. The metrical hierarchy supplies the shape and the harmonic rhythm supplies the height, and between them they turn a row of bars into a curve with a value at every instant rather than at every event.

The two scale together, so the ratio between them is fixed at 6.7 by the metrical weights and does not move with the rate at all. How often the harmony moves sets the height of the curve; where in the bar it is likely to move is set by the metre and by nothing else — which is what makes the two ingredients separable in the first place.

The survival term, and what it is for #

There is a third quantity in the construction and it is the one that makes the curve a curve rather than a profile.

The survival is the probability that the chord has not changed by a given beat, and it falls through the bar because each beat that passes without a change uses up some of the probability. So the hazard multiplied by the survival — the unconditional chance of a change arriving at a beat — is not the same shape as the hazard itself: it is the metrical profile tilted downward through the bar.

That is the shape a listener’s expectation actually has if the model is right, and it has a musically sensible consequence. A change late in a bar is more surprising than the metrical weight alone would say, because by then the listener has had several beats’ worth of evidence that this chord is staying.

It also means the second bar of a two-bar phrase looks different from the first, because the survival resets whenever a change arrives. Two bars with a change on the first downbeat only are not two copies of one bar; the second one’s expectations are lower throughout.

What the identity term looks like beside it #

Setting the two surprises side by side is the point of having both, and the comparison has a shape.

The identity surprise of a chord in this collection’s model runs from under a bit for a dominant arriving after a tonic to five or six bits for a chromatic third relation nobody was expecting. That is a range of roughly five bits.

The timing surprise runs from 1.8 bits on the downbeat at one change a bar to 4.5 on the weakest position — a range of under three.

So the identity term is the larger of the two on this repertoire, by about a factor of two, and the timing term is not negligible beside it. A chord that is unremarkable in itself and arrives in an unremarkable place costs about two bits; the same chord arriving on the last quaver costs about five, which is the price of a genuinely unexpected harmony arriving on the beat.

That is the practical content of joining them: a placement can be worth as much as a chord, and a composer working with an ordinary harmonic vocabulary has a second axis of surprise available that costs nothing in the notes.

Which computation produced the numbers #

The metrical weights are a standard hierarchy for 4/4 — 1, 0.15, 0.5, 0.15, 0.85, 0.15, 0.5, 0.15 over eight positions — with the third-beat weight below the first and the fifth position, the halfway point, close behind the downbeat. They are asserted, and they are the profile every metrical model in the literature uses in one form or another.

The hazard is that profile scaled so that the sum over a bar equals the harmonic rhythm in changes per bar, and capped at 0.95 so that a saturating profile stays a probability.

The timing surprise is −log₂ of the hazard at the beats where a change arrives, summed. The flat control is the same sum with every hazard equal to the rate divided by the number of positions, which is the reading a listener with no metrical hierarchy would get.

The two bars are not two copies #

The survival term has a consequence that is easy to miss and easy to hear.

In a two-bar unit with the harmony changing only on the first downbeat, the first bar’s beats are met with a survival of one, falling as the bar goes on. By the second bar the survival has fallen further, because seven more beats have passed without a change — so every hazard in the second bar is multiplied by a smaller number, and the unconditional expectation of a change is lower everywhere in it.

Then a change arrives, and the survival resets to one.

What that describes is a listener whose expectation of movement decays while nothing moves and is restored when something does. It is the same shape as the running impression of loudness on the loudness ladder — a quantity with a memory that resets on an event — and it is the reason a long static harmony feels increasingly like a preparation.

It also predicts something specific: a chord held for four bars and then changing on the downbeat of the fifth produces a larger timing surprise than the same change after one bar, even though the beat it lands on is equally strong. The metre says the beat was likely; the survival says the change was overdue and therefore, by this model, less expected rather than more.

That is a prediction the model makes and a listener probably contradicts. A long-delayed arrival is usually described as more expected as it approaches, not less — which is a real problem for the hazard as a model of anticipation, and is the kind of thing a hazard rate estimated from a corpus rather than assumed would fix, since a real repertoire’s chord durations are not memoryless.

Where the model stops #

The weights are asserted. Everything about the shape follows from them, and a different published hierarchy would give a different ratio — though every one of them has the downbeat highest and the off-beats lowest, so the direction of the syncopation result is robust.

The harmonic rhythm is a constant. How often the chord changes is a rate measured over a passage, and a real passage’s rate varies: it accelerates into cadences and slows in the middle of phrases. A hazard that used a local rate would be a better model and would need a corpus.

Nothing here knows what the chord is. The timing surprise is about a change arriving and says nothing about which one, which is the point — the identity term is the ladder’s existing business. Whether a listener’s two expectations are independent, or whether an unlikely chord is also expected at an unlikely time, is a question about a corpus.

And there is no phrase. A listener’s expectation of a change is enormously higher at a cadence than in the middle of a phrase, and nothing in a bar-level hazard has a phrase in it.

What the picture cannot show #

It cannot show a listener. Everything here is a model of what an ideal observer with the right statistics would expect, and whether a listener’s expectation follows a hazard is a question for an experiment — of which there are many, and all of them measure something rather different.

Nor can it show learning. Counting produced the hierarchy is this ladder’s account of where the profile comes from — exposure to a repertoire — and the metrical weights here are asserted rather than counted. They could be counted, from the same corpus that would give the local harmonic rhythm.

It cannot show the segmentation. Which notes are the chord shows that deciding where one harmony ends and the next begins is itself a decision that depends on the metre — so a hazard indexed by metrical position is using an analysis the metre already helped produce, and the circularity is real.

And it cannot show the anticipation. The single most common expressive placement of a harmony in tonal music is early — the chord arrives on the last part of the previous bar — and it is common enough that a listener may well have a separate expectation for it. A model with one hazard per metrical position cannot represent a systematic pre-placement.

Whose music, and when #

The metrical weights and the harmonic rhythm are both drawn from common-practice European tonal music of roughly 1700 to 1900, which is the repertoire the whole of this ladder is about and the one the probe-tone profiles were measured on.

The claim about syncopation is narrower than it looks. In that repertoire a harmonic change off the beat is an expressive event and is treated as one. In a great deal of music since — and in a great deal of music from elsewhere — harmonic change is not aligned with a metrical hierarchy of this shape at all, and in some traditions there is no harmonic rhythm to align.

The construction would transfer. Any tradition with a metrical hierarchy and a rate of harmonic change has a hazard, and the interesting comparison is what the ratio of strongest to weakest comes out as in a tradition whose metre is not a nested duple hierarchy — which is a question the additive-metre ladder has the metres for and this collection has no counts for.

Where this ladder goes next #

Five rungs. Counting produced the hierarchy; a chord did not come and the moment was described; the moment acquired a number; the number acquired an uncertainty; and now the number has a companion, for when rather than for which.

What is owed after this is the sum. There are two surprises available at every chord change and a listener meets one event, so the honest quantity is the total — and adding them requires knowing whether they are independent, which is a fact about a repertoire rather than about a model. A corpus of harmonic analyses with bar positions in it would settle it in one pass, and it is the same corpus the key ladder and the progression ladder have each recorded independently. Three ladders now want one thing.