The surprise of nothing happening

Two surprises and one event added the two quantities a listener meets at a chord change — which chord it is, and when it came — and ended by naming a third:

Both quantities here are about a chord change, and a listener meets a great deal of music in which nothing changes. Summed over a bar it is a running quantity of the same kind as the two here, in the same units, and it is much larger than either because there are more beats than changes.

The term exists and has been computed since expectation is a curve, not a list built the hazard. A hazard is the chance the chord changes on this beat given that it has not changed yet, and a probability charges twice: minus the log of it when the event happens, and minus the log of one minus it when the event does not.

The second charge has never been spent. Spending it gives the number the debt predicted, and the number is not the one the debt predicted.

A third, not most #

In a bar of eight quavers at one chord change a bar, the hazards run from 0.290 on the downbeat to 0.043 on the weak quavers. Each beat charges its arrival with probability equal to that hazard and its non-arrival with the rest.

Over the bar the arrivals come to 2.61 bits and the non-arrivals to 1.29. The term the ladder had never spent is 33 per cent of the total rather than most of it.

The prediction was reasonable and the arithmetic overturns it, and the reason is worth stating carefully because it is the whole of the essay.

There are indeed more non-events than events — seven beats against one at this rate — but a non-event is nearly free. Where a hazard is 0.043, being surprised by an arrival costs 4.52 bits and being surprised by the absence costs 0.064. That is not a small difference; it is a factor of seventy. Minus the log of one minus a small number is approximately that number over the natural log of two, so as the hazard falls the arrival cost rises logarithmically and the non-arrival cost falls linearly.

Seven beats at 0.064 do not add up to one beat at 1.79. They add up to about a quarter of it.

The sum is an entropy, which fixes the total #

Adding the two terms at a single beat, each weighted by its own probability, gives exactly the binary entropy of that beat’s hazard. That is not an approximation and not a coincidence — it is what an entropy is.

So a bar’s whole timing bill is the sum of the binary entropies of its beats’ hazards, and at one chord a bar in four it is 3.90 bits. Nothing about where the changes actually fall can raise or lower it: the schedule of changes decides how the bill is split between the two terms and not how large it is.

That is a conservation law and it is the first one this ladder has had. Every quantity above it has been a sum over events, which a passage can have more or fewer of. This one is a sum over beats, which a bar has a fixed number of.

The binary entropy is largest at a hazard of one half and falls away on both sides. So the timing information available in a bar is maximised when the chord is as likely to change on each beat as not — and no music does that, which is why the numbers here are all a long way below the ceiling.

Why a term this large was invisible for six rungs #

The hold cost is not obscure and it is not expensive to compute. It has been sitting in the curve’s own output since the curve was built, next to the arrival cost, computed on the same line. Nobody spent it, and the reason is structural rather than an oversight.

Every quantity on this ladder before the fifth rung is a list with one entry per event. Surprise is a number gives a bit-count to each step of a progression; a chord, given a key and a predecessor makes each of those conditional on more; the chord that did not come is about an expectation defeated at a particular moment. A list indexed by events has nowhere to put a quantity that is paid when there is no event.

The fifth rung changed the representation from a list to a curve and did not change what was read off it. The curve was drawn across the bar and then sampled at the marked beats, which is a list again.

A representation decides which questions are askable, and this one made a term worth a third of the bill unaskable for as long as the ladder used it. That is not a fact about harmony. It is the same shape as the metre ladder’s beat that is never sounded, where the thing that mattered was the position nothing happened at.

On the path a listener actually walks #

The expectation is one way to state it and it is not what a listener meets. A listener meets one bar with the changes where they are, and the bill on that path is a different arithmetic.

Put the single change on the downbeat, which is where every figure of this ladder puts it. The listener pays 1.79 bits for the arrival and then pays the hold cost at the seven remaining beats, which comes to 1.12. So on the realised path the non-events are 38 per cent of the timing bill — a little more than in expectation, because the arrival landed on the cheapest beat there is and the waiting was charged at full price.

Move the change to the last quaver of the bar and the split inverts in emphasis rather than in size, because the seven beats of waiting are now charged before an arrival that costs 4.52. That case is where the chord actually lands, and it turns out to be a rung of its own.

Either way the answer is the same shape: the non-events are a real and reportable third of the bill and are not the bulk of it.

Where the term does grow #

The share is not fixed. It depends on the harmonic rhythm, and it depends on it monotonically, which makes it a clean thing to report.

At an eighth of a chord a bar — a slow chorale, one harmony every eight bars — the non-events are 20 per cent. At half a chord a bar, 28. At one, 33. At two, 41. At four, 46.

It passes a half at eight changes a bar, which is one change on every quaver of a bar of four. At that rate there is no harmonic rhythm left to be surprised by: a chord that changes on every available position is not a chord that changes.

So over the whole range in which a metre is audible, the events carry more than the non-events, and the crossing sits outside music rather than inside it.

The reason for the monotone rise is the mirror of the reason for the small starting value. As the hazard rises the arrival gets cheaper — a change is unsurprising when changes are frequent — while the non-arrival gets dearer, because a beat that was very likely to carry a change and did not is genuinely informative. The two move in opposite directions and cross where the hazard passes a half.

Which of the two dominates is a fact about the repertoire #

That gives the ladder something it wanted from the corpus it does not have and can get without one.

How often the chord changes put a range on harmonic rhythm and found its limits set outside music — by how fast a listener can integrate a chord at one end and how long a memory holds one at the other. Everything on this ladder is computed at one chord a bar, near the middle of that range.

Taking the range seriously, the non-event share runs from about a fifth at the slow end to about a half at the fast end. A chorale and a fast harmonic texture are not the same listening problem, and the difference is not that one has more surprise in it but that the surprise is in a different place: in a slow texture almost all of a listener’s timing information arrives when something happens, and in a fast one nearly half of it arrives while nothing does.

That is a statement about attention with a consequence. A listener following slow harmony can afford to attend at the changes; a listener following fast harmony cannot, because half of what there is to know is in the beats between them.

It also puts a number on something the collection has said qualitatively. A cycle cannot cadence argues that a repeating pattern with no change in it has no closure available to it, and this arithmetic says what such a pattern still carries: a passage in which the chord never changes at all pays no arrival cost and the full hold cost of every beat, which at one nominal change a bar in four is 1.61 bits a bar. A texture with no harmonic events in it is not silent to a listener counting bits. It is a running statement that the thing which was likely to happen has not.

And at the other end the shape agrees with what the bar above the bar says about hypermetre. If a listener’s hazard is defined over a two-bar or four-bar unit rather than a one-bar one, every hazard falls, every arrival gets dearer and every silence gets cheaper — so the non-event share falls with the level at which the metre is read. A listener hearing in four-bar units is a listener for whom nearly all the timing information is at the changes.

The metre moves it too, and in the direction that is easy to guess wrong #

Run the same arithmetic in three and the non-event share rises to 36 per cent at one chord a bar, from 33 in four.

That looks backwards at first — a bar of three has fewer beats to wait through than a bar of four, so there ought to be less waiting. The answer is that perBar is the expected number of changes in a bar however many beats it has, so a bar of six quavers spreads the same expected change over fewer positions and every hazard is higher. Higher hazards mean dearer silences.

The general statement is that the non-event share is set by the hazards and not by the count of beats, and the two move together only when the harmonic rate is held per beat rather than per bar. Held per bar, a shorter bar has more surprise in its silences.

Whether harmonic rhythm is a per-bar or a per-beat quantity is a question this collection has not settled and the metre ladder’s account of the beat’s preferred rate bears on: if what a listener holds constant is the interval between changes in seconds, then neither is right and the bar is the wrong unit for all of it.

What this does to the two surprises #

The sixth rung reported a mean total of 5.78 bits per chord change at the independent end of its family and 3.79 at the dependent end, and said the missing correlation was worth a factor of 1.53.

The third term does not enter that number, because it is not paid at a change. It is paid alongside it, and the honest per-bar quantity is now three things rather than two: the identity surprise of whatever changes did happen, the timing surprise of when they happened, and the hold cost of every beat that passed without one.

At one chord a bar in four that is roughly 2.3 bits of identity, 1.8 of timing and 1.1 of holding — so the term this essay adds is a fifth of what a listener meets in a bar, and it was previously counted as zero.

There is one further thing the three-term accounting fixes, and it is a small honesty rather than a finding. A per-change quantity cannot be compared between passages of different harmonic rhythm, because a passage with more changes has more of them to pay for. A per-bar quantity can. Both terms here are per bar by construction, and the identity surprise can be put on the same footing by multiplying it by the rate. The ladder has been quoting bits per change and bits per bar as though they were the same units, and they are not.

And it does not need the correlation. The hold cost comes out of the metrical hazard alone, which is a quantity this ladder computes rather than assumes, so it carries none of the fifty-per-cent bracket the other two carry. The sizes above are comparable in the sense that they are all bits, and comparable in no stronger sense than that until somebody has counted the joint distribution.

Which computation produced the numbers #

The hazard is the fifth rung’s, unchanged: the metrical weights scaled so that the expected number of changes in a bar comes out at the stated harmonic rate.

The arrival cost is minus the base-two logarithm of the hazard and the hold cost is minus the base-two logarithm of one minus it. The per-beat expectation weights each by its own probability, and the figures assert that the two together are the binary entropy of the hazard, which is the check that no beat is being counted twice or missed.

The metrical weights are Longuet-Higgins and Lee’s, which is where syncopation is a number about the metre gets its own arithmetic, so the shape of the curve is shared with the metre ladder rather than invented here.

The identity surprise quoted at the end is the fourth rung’s, from the root-motion frequencies, and is unchanged.

Where the model stops #

A hazard is not a listener. It is a probability the model assigns, and nothing here says a listener’s expectation of a chord change on the third quaver of a bar is 0.145 or anything else. Counting produced the hierarchy is this ladder’s one measured quantity and it is about pitch, not timing.

The beats are independent and they are not. The hazard treats each beat as a fresh coin given that nothing has happened, which is what a hazard means, but a real harmonic rhythm has periodicity in it — a passage that changes every two bars keeps changing every two bars — and a model with that in it would charge much less for the beats it has learned to expect nothing on.

Nothing here knows what did not change. The hold cost is the cost of the chord staying, and a chord staying is not one event: an unchanged harmony with a moving melody over it is a different listening experience from a held chord, and both cost the same here.

And the cap binds at the top of the sweep. Above about two changes a bar the downbeat hazard is clipped so it cannot exceed a probability, and the numbers at eight and sixteen changes a bar are shaped by that clipping as much as by the metre.

Where this ladder goes next #

Seven rungs: a measured hierarchy of pitch stability, a chord that fails to arrive, a surprise given a number, that number made conditional on a key as well as a predecessor, expectation drawn as a curve instead of a list, two surprises added under a swept correlation, and now the third term the curve had all along.

What is owed after this is where the change lands. Every hazard on this ladder has been evaluated at a downbeat: the curve is drawn across the whole bar, the ratio of six and a half between its ends is quoted, and then every number is taken at the peak. A change that arrives on the last quaver of a bar has been preceded by seven beats of waiting the listener has already paid for, so the honest cost of an arrival is the waiting plus the arrival — and written that way the costs over the bar become a distribution rather than a list of surprises, with the mass left over sitting on the case nobody has priced at all, which is a bar in which the chord does not change. That needs no corpus and no listener. It is the same curve, read at the beats the ladder has been stepping over.