Why not mix arm/intel in conda environment?

336 Views Asked by At

Many questions ask how to get M1/M2 apple silicon macs to install pure Intel or pure arm conda environments.

There are good answers on how to achieve both configurations.

The question I have not seen answered anywhere after extensive search is why it is not possible to mix architectures in one environment.

Is this an example of dynamically loaded library (DLL) hell?

If I just want to mix binaries, there should be no problem.

I should be able to use whichever architecture is more performant. Rosetta 2 should do emulation based on the binary architecture.

Where in the dependency chain do the problems arise? Can I ignore the warnings and still install mixed packages in the hope that they behave like separate binaries?

1

There are 1 best solutions below

0
On BEST ANSWER

Yeah, you can start a separate process of the other architecture, and talk to it through pipes or maybe even shared memory. But not calling into a library.

There's no calling convention for that, no way of figuring out which x86 registers should get what value after a jump or call from ARM64 code, or vice versa. (And in the general case, that could be a non-solvable problem for variadic functions, with x86-64 System V I think passing more FP args in registers than ARM64, and you wouldn't know which stack args to load into FP registers in a trampoline/shim/wrapper that would try to adapt between them, if that even makes sense because the callee would also be native ARM64 code that's just compiled from x86-64 machine code. You don't actually have x86-64 registers, although ARM64 does have more registers of the same width as the integer and vector regs Rosetta-2 supports (not AVX))

The emulation is at a whole-process level, for all of virtual address space of a whole process. Not on a per-function level to port individual functions to the standard ARM64 calling convention. Different ABIs (sizes and alignments for types, and struct layout) is also a problem.

If Rosetta-2 isn't fully ahead-of-time recompilation, i.e. if there's any run-time component like how a JVM JIT compiles but still runs the code inside a VM, then that would also have to start up when calling from an ARM64 process into an x86-64 function.