aarch64 MMU translation table

Question

I've been experimenting with MMU on RPi3B+ with no success at all. I tried following official ARM examples, and I generate tables like this:

Level 2 table is at 1016000 PA:

0000000001016000: 0000000000000405 0000000000200405 0000000000400405 0000000000600405
0000000001016020: 0000000000800405 0000000000A00405 0000000000C00405 0000000000E00405
0000000001016040: 0000000001000405 0000000001200405 0000000001400405 0000000001600405
...
0000000001016FC0: 006000003F000601 006000003F200601 006000003F400601 006000003F600601
0000000001016FE0: 006000003F800601 006000003FA00601 006000003FC00601 006000003FE00601

Level 1 table is at 1014000:

0000000001014000: 0000000001016003 0000000000000000 0000000000000000 0000000000000000
0000000001014020: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
...
0000000001014FE0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000

The only entry in Level 1 table is the one that points to a Level 2 table.

MAIR: 0x000000000044FF00
TCR: 0x0000000000803520

Anyway, when I enable MMU, I lose any other information, as UART does not work anymore. This should be 1:1 mapping in EL1. I tried experimenting with both 32bits and 39bits VA space in TCR. Also with enabling ttbr1, nothing works at all. What am I doing wrong?

Edit: This works perfectly in qemu-system-aarch64 -M raspi3, but not on device when using U-Boot. Also it does not work if loaded directly in RPi (at 0x80000). Any explanation as to why is welcome.

Answer 1

It seems the problem was somewhere in my code for enabling / disabling MMU. Because I wanted to keep the same logic in early start and when enabling MMU, I wrote this macro:

.macro set_mmu_el el enabled
    mov   x0, xzr

    ldr     x2, =\enabled
    lsl     x1, x2, #12
    orr     x0, x0, x1
    lsl     x1, x2, #2
    orr     x0, x0, x1
    orr     x0, x0, x2

    msr     sctlr_el\el, x0
.endm

Then, this was called in:

.globl mmu_enable_el1
mmu_enable_el1:
    sub     sp, sp, #0x16
    str     x30, [sp]

    dsb ish
    isb

    set_mmu_el 1 _MMU_ENABLED
    isb

    nop
    nop
    nop
    nop

    ldr     x30, [sp]
    add     sp, sp, #0x16

    ret

And again, this was called in:

void mmu_init() {
    // ...
    mmu_enable_el1();
}

I replaced this with only:

asm volatile("dsb ish; isb; msr sctlr_el1, %0; isb; nop; nop; nop; nop"::"r"(0x000005 | (1 << 12)));

Now it works perfectly, with both D-cache and I-cache enabled. I am not sure what went wrong with the code above, but anyway, a lesson has been learned. At least my page table format was correct.