Sure, it may not be precise repetition, but my idea here is that none of B', B'', etc. are really different than B (they may even compile down to the exact same bytecode), they're just the same thing but written differently. And in fact, none of these are really faster than A, even if they're all "changes". But it's the same issue as any other form of p-hacking, where you keep trying more and more trivial B-variations until you eventually get the result that you're looking for, by random chance. (Cf. the example in xkcd 882, which does change the experimental protocol each time, but only trivially.)

There is, in fact, "something wrong" with this, which is what GP was pointing out. It's literally covered under "Playing with multiple comparisons" in TFA.

(Personally, to combat this, I've ignored the fancy p-values and resorted to the eyeball test of whether it very consistently produces a noticable speedup.)