The questions stems from reading the Spectre attack paper. If I understand it correctly the attack stems from the possibility of CPU heuristics speculatively executing (the wrong) branch of code. Consider the example (in C):
int arr[42];
if (i < 42) {
int j = arr[i];
}
If I understand the paper correctly, the int j = arr[i] can be (in certain circumstances) speculatively executed even when i >= 42. My question is - when I access array outside of its bounds my program would often crash (segmentation fault on Linux, "The program performed an illegal operation" error on Windows).
Why does speculative execution not cause programs to crash in case of array out of bound access?
The key point is that in modern CPUs the verb executing doesn't mean what you think it means.
To execute an instruction is the act of computing its output and side effects if any.
However, this doesn't change the program state.
This seems hard to grasp at first but it's really nothing exotic.
The CPU has a quite big internal memory made by all its registers, most of this memory is not visible to the programmer, the part that is it is known as the architectural state.
The architectural state (AS) is what is documented in the CPU manuals and what is altered by a sequence of instructions (a program, for example).
Since altering the AS can only happen with the semantics given in the ISA (the manuals) and the ISA specify a serial semantics (instructions are completed one after the other in the program order) this doesn't allow parallelism.
However, a modern CPU has a lot of resources (known as execution unit) that can do their work independently.
To exploit all these resources the front-end of the CPU (the part that is responsible for reading instructions from the memory hierarchy and feeding them to the execution units) is able to reach, decode and output more that one instruction per cycle.
The boundary between the front-end and the back-end (where the execution units lie) is not really dealing with instructions anymore (but with uops) but that's an x86 CISC nuisance.
So now the CPU is given 4/6 uops to "execute" at a time but if the ISA is serial, what it could possibly do other than queuing these uops?
Well, the front-end is made so that these uops don't operate on the AS but on a shadow state (SS, my terminology here), their operands are renamed, made of part of the big invisible memory of the CPU.
Altering the in parallel or out-of-order is fine as it is not the AS.
This is what execution is: altering the SS.
Does it really worth it? Afterall it is the AS that matters.
Well, transferring the SS to the AS is really fast compared to execution, so it's worth it.
It is a matter of "renaming back" (inverting the previous renaming) and it is called retiring of the instructions.
Actually, retiring is a bit more than that.
Since execution doesn't affect the AS, the side effect should also not affect it.
These side effects include exceptions but speculatively handling an exception is too cumbersome (it needs to coordinate a lot of resources) so exception handling is delayed until retirement.
This also has the advantage of having the correct AS at the moment the exception is handled and the advantage of raising an exception only when it must actually be.
The point of speculative execution is to bet, the CPU bets that the instructions sequence doesn't generate any exception (including page fault) and thus execute it with most checks off (I cannot exclude, off the top of my head, that some check is not made regardless) thereby gaining a lot of advantage.
When it's time to retire those instructions the bets are checked and if any fails, the SS is discarded.
That's why speculatively execution doesn't crash your program.
What Spectre relies on is the fact that speculatively execution does indeed alter the AS in some sense: the caches are not invalidated (again for performance reasons, the SS is simply not copied into the AS when a bet is off) and timing attacks are possible.
This could be corrected in a number of ways, including performing a basic privilege check when reading from the TLB (after all only privileges 0 and 3 are used, so the logic is simple) or adding a bit to the cache lines to mark them speculative (treated as invalid by non speculative code).