How do I use microprogramming to modify the instruction set architecture of an Intel CPU?

Question

Suppose I wanted to modify the implementation of an existing instruction. For instance, suppose I wanted to change the microcode that is executed by my processor when a "mov" instruction is executed after being fetched and decoded. How would I do that?

Also, suppose I wanted to add a new instruction to the ISA. For instance, suppose there is an opcode that isn't currently being used that I wanted my processor to be able to decode and execute after it is fetched. How would I do that?

Truthfully, I don't even know where to begin with this, which is why I'm asking on stack overflow. I don't have any experience writing microcode, and I only know the term from my computer architecture class. I know it's pretty easy to look up x86 instruction listings, but I don't know where I would even look up microinstruction listings. I don't even know if Intel makes their microcode listings public. I don't know what program I would need to use to load those new microprograms into the appropriate memory inside my CPU. I think the bootloader might be involved, but I could be totally wrong about that. If I own my CPU, I think I am legally able to modify its microcode and reverse engineer its instructions, but it would be easier to understand how my CPU works and how its microcode can be altered through existing documentation, rather than through reverse engineering.

I know there are some smaller hardware vendors that make open source ISAs--so maybe buying a CPU from one of those vendors would be good if I want to experiment with this? But I don't know what CPU from an alternative vendor would be the best for that. But really, I don't think buying a new CPU should be necessary for what I'm trying to do, since I could just use my existing CPU to run an open source ISA in a virtual machine if I had to (I'm interested in microinstructions and microprograms from a software perspective, not a hardware perspective).

Answer 1

For instance, suppose I wanted to change the microcode that is executed by my processor when a "mov" instruction is executed after being fetched and decoded. How would I do that?

You'd:

a) Get a master's degree in Computer Engineering, Computer Science and/or Electrical/Electronic Engineering.

b) Apply for a job at Intel (or AMD), and get it

c) Work your way up the corporate structure to reach a position where you're able to influence the design of future products

Note that a simple mov instruction wouldn't use micro-code, so you'd have to change that first.

I don't have any experience writing microcode

Microcode is encrypted and treated as a trade secret; so "almost nobody on earth" (I'm guessing maybe 20 Intel employees?) has experience writing microcode for Intel CPUs.

I know there are some smaller hardware vendors that make open source ISAs--so maybe buying a CPU from one of those vendors would be good if I want to experiment with this?

"Open source ISA" mostly just means that a huge company that's able to spend many millions of $$ to manufacture their own chips saves about 50 cents on licensing fees.

The practical alternative is to use an FPGA (but the cost and performance is nowhere near what you'd get from a custom ASIC).

Answer 2

The various mov opcodes decode to a single uop in modern x86 CPUs (https://uops.info/ https://agner.org/optimize/), in fixed-function logic that can't be modified by microcode. (Except mov to segment register, or mov to/from control or debug registers.)

---

Even Intel couldn't add arbitrary new instructions when they wanted to for Spectre mitigation; some of the decoder hardware is fixed-function, producing uops directly, only indirecting to the microcode sequencer for certain opcodes that need more than 4 uops.

What Intel did was add a new MSR (model-specific register). wrmsr takes a number in ECX, and uses that number to figure out which microcode to run. Like a "function pointer" except for the microcode. So this new MSR was a way that a microcode update could add some new microcode to an old CPU, so OSes could get it to run and modify the CPU's internal state.

So ECX=49H (IA32_PRED_CMD) / wrmsr acts like a new instruction which you run for the side-effect, not to actually set a value. See https://www.intel.com/content/dam/develop/external/us/en/documents/336996-speculative-execution-side-channel-mitigations.pdf

---

Similarly, to allow OSes to mitigate some MDS vulnerabilities by cleaning internal state at certain points, Intel added to the behaviour of verw, instead of adding a different instruction. See https://security.stackexchange.com/questions/210209/what-are-the-new-mds-attacks-and-how-can-they-be-mitigated

Possibly some of the motivation was that the coding space for 32-bit opcodes is already full, but equally likely that it was already a microcoded instruction: More than 4 uops so it was handled by indirecting to the microcode sequencer. So microcode updates can replace the sequence of uops that gets issued.

(64-bit mode has quite a few unused opcodes, as AMD64 removed some obsolete instructions like aaa and push/pop of segment regs.)

---

I don't even know if Intel makes their microcode listings public.

Very much no. It's not only undocumented, the microcode blobs are encrypted and signed with a secret key.

The key for Goldmont has leaked, so its firmware can be decrypted and potentially reverse-engineered, although it won't be easy with zero documentation of the format or what the bits mean. But still no way to load modified firmware into a real CPU, AFAIK.

Answer 3

If you are interested in custom instructions, check out RISC V.  It has opcodes reserved for customization, and being open source (hardware), can be edited without license or joining some private organization.