Newton attention

Each card becomes a phase per channel, and pooled powers of the phases count relations of every size. One layer on four-card tables, opened up: which characters of the deck its channels carry, and which arity it relies on.

Characters found One square per character, up to conjugation. White: some channel is it.

40 of 40

The ideal layer is built with every character, so nothing is learned over time. Train a layer live to watch the count rise.

Phases on

χ = (1, 0, 0, 0)

The ideal layer's channel for this character: three exact phases

Dots: the 81 cards, shaded by the character's value, stacked where phases coincide. Cards: this table.

Arities it uses

Held-out accuracy with one arity's term switched off. A four-card relation needs arity 4.

k = 1

100.0%

k = 2

100.0%

k = 3

100.0%

k = 4

53.5%

k = 5

100.0%

Bars run from always answering the majority (53%) to 100%; the tick is the full layer, 100.0%.

This table

A card is positive when it and three others sum to zero. Bars: the layer's probability; a dot marks the true answer. Pick a card to follow it on the wheel.

Ideal layer

Built by hand, not learned: one channel per character of the deck, and a read-out of the arity-4 count alone, the construction in the paper's Newton's shortcut section. Recorded and Live show what gradient descent finds.

R = \frac{\sum g e ^{i (θ - 2 π ⟨ c, x ⟩ /3)}}{\sum g}

How close a channel is to the character $c$ , over every card at every position. A square counts as found above 0.9.