How many instructions can the M6 keep in flight?

A core doesn't run instructions strictly in order. It looks ahead through a window of instructions that have been fetched but not finished, and runs whatever is ready. How big that window is decides how much a slow load can hide. I measured it on the same chip as the cache and branch predictor posts. The short version: a Super core can have about 1,050 instructions in flight at once. Inside that, it runs out of integer results at about 400 and floating-point results at about 880, whichever the code hits first, and nops don't seem to count. An Efficiency core's window is about 185, so the Super window is roughly 5.7 times bigger. A Performance core sits in between, with about 210 integer and 330 floating-point results.

The measurement

The idea is Henry Wong's; the setup here is mine. The program generates a loop at run time with two independent random pointer chases over 256 MB each, so every load misses to DRAM, about 90 ns. Each iteration is: load A, N filler instructions, load B, N more filler instructions. If load B fits in the window behind load A's fillers, the two misses overlap and an iteration costs about one DRAM latency. Once B no longer fits, the core can't see it until A retires, the misses run one after the other, and the iteration costs about two. The N where the time jumps is the size of whatever resource the filler uses up.

The fillers:

• add x7, x8, #1 needs an integer rename register for its result.
• fadd d3, d4, d5 needs a floating-point one.
• nop needs no result register.
• Mixes in a repeating pattern: mix31 is 3 adds to 1 fadd, mix13 is 1 add to 3 fadds, mix12 is add, fadd, fadd, mix11 alternates add and fadd, and addnop alternates add and nop.

macOS can't pin a thread to a core, so I used the same check as the other posts. Every measurement is bracketed by an 80 KB pointer chase and a dependent-add clock reading, which together identify the tier. Rows whose before and after fingerprints disagreed were dropped, the rest were binned by the tier they actually ran on, and I took the median per N. The Super rows ran at about 4.4 GHz and the Efficiency rows at about 2.7-2.8 GHz.

Super cores

Each filler gets a sharp step, at a different place:

• add: 107.6 ns at N=384, 143.5 at 400, 189.5 at 416. About 400 integer results in flight.
• fadd: 135.4 ns at N=880, 173.5 at 896, 192.6 at 912. About 880 floating-point results, a bit over twice the integer number. The line also creeps up from about 90 ns at N=432 to 135 before the step, which I haven't explained.
• nop: flat at about 90 ns up to N=1650, then a straight line through the origin: 110 ns at 2000, 214 ns at 4000. That's the front end's rate. Two blocks of N nops is 2N instructions per iteration, so 8000 in 214 ns at about 4.4 GHz is roughly 8.5 per cycle. No window limit shows up before throughput takes over, so as an inference, nops don't take a slot that matters here.

The mixes

The mixes are what make this read as two limits rather than one. Count the adds at each step:

• mix31 (3 adds to 1 fadd) steps between N=512 (116 ns) and 544 (194 ns), with 144.7 at 528. Three quarters of about 530 is about 400 adds.
• mix11 (add, fadd) steps between N=784 (125.6) and 816 (197.6), with 168.3 at 800. Half of 800 is 400 adds.
• addnop (add, nop) steps in the same place, 130.0 at 784, 178.5 at 800, 197.9 at 816. Again 400 adds, and now the other half is nops.

Whatever sits between the adds, fadds or nops, the step lands where the adds reach about 400. As far as these fillers go, the integer limit is independent of what else is in the window.

The mixes with more floating point hit something else. mix13 (1 add to 3 fadds) is at 140 ns at N=1024 and 156 at 1040, then 210 at 1056. At the step that's about 790 fadds and about 260 adds, and neither is at its own limit (880 and 400). mix12 (add, fadd, fadd) is the sharper test: adds alone would allow N of about 1200 and fadds alone about 1320, yet it steps at the same place, 124.1 ns at N=1024, 137.5 at 1040 and 191.3 at 1056.

So there's a third, shared cap of about 1,050 instructions in flight, whatever mix they are. That's the window itself. My guess, and it is a guess, is that it's the reorder buffer, the structure that holds every instruction from dispatch until it retires. The picture for a Super core is then roughly 400 integer results, 880 floating-point results, and 1,050 instructions in total, with whichever comes first deciding where the step lands. Pure adds hit 400 first, pure fadds hit 880 first, and a mix with under about 40% adds and under about 85% fadds reaches 1,050 before either.

The nops stayed flat to N=1650, which is more than 1,050 of them in the window. So either nops don't count toward the total or they leave it early. I can't tell which from this test.

Efficiency and Performance cores

On the Efficiency cores both add and fadd step in the same place, between N=160 and 192: add goes 128.8, 162.2, 220.2 at N=160, 176, 192, and fadd goes 140.6, 165.9, 221.6. An add and fadd 1:1 mix does the same: 151.1 ns at N=160, 188.4 at 176, 257.8 at 192. So for add, fadd and a 1:1 mix the Efficiency limit lands between N=176 and 192, call it about 185. The FP-heavy mixes (1:2 and 1:3), run at the lower clock, are too noisy to place exactly, but they also break by N=176-192. That makes it one shared window that fills before any integer or FP limit shows up. If the Efficiency cores have separate register limits, they sit above 185, where this test can't see them. Nops don't hit it: that curve is flat until about N=450, though those rows ran at a lower clock (about 1 GHz), so where it bends is probably throughput, not a window. These rows are noisier than the Super ones; after the step the pure add and fadd scatter between about 300 and 500 ns rather than settling.

Update, same day: the Performance cores took a second try. Leaving the thread's QoS unset put a share of the runs on them (anywhere from a few percent to about half, depending on what else was running), with two busy-loop hogs added to fill the Super cores when it was low, and the fingerprint kept only rows that started and ended there (about 4.3-4.4 GHz). The 80 KB fingerprint misses their smaller L1, which is what tells them apart from a Super core at the same clock. They look like a scaled-down Super core: separate integer and FP limits, both much smaller.

• add: 115.5 ns at N=200, 120.5 at 208, 169.5 at 216, 204.0 at 224. About 210 integer results.
• fadd: 133.0 at N=320, 132 at 328, 174 at 336, 202 at 344. About 330 floating-point results.
• mix11 (add, fadd): 127 at N=416, 143 at 424, 188 at 432. Half of about 425 is about 212 adds, the integer limit again.
• mix13 (1 add to 3 fadds): 135.5 at N=432, 147 at 440, 188.5 at 448. Three quarters of about 444 is about 333 fadds, the FP limit again.

Every step lands where one register type runs out, so the shared window on a Performance core is at least about 445 and this test never reaches it. Filling it would need a filler that uses neither register type and still counts, which the nops don't seem to.

What I'm unsure about

• What the structures are. The filler decides which resource runs out first, so I'm reading each step as the size of whatever that filler uses up. "Integer registers", "FP registers" and "reorder buffer" are the natural names, not something I've isolated.
• Nops. They look free, but the test can't say whether they skip the window or leave it early.
• The slow creep before the fadd step and the noisy Efficiency rows. The medians hold up, but individual points don't.

The takeaway: a Super core can look about 1,050 instructions past a DRAM miss, an Efficiency core about 185, and a Performance core runs out of registers at about 210 integer or 330 FP results before its window fills. Within the Super window, integer-heavy code runs out of room first, at about 400 results, while FP-heavy code gets to about 880.