How is a spin lock woken up in Linux/ARM64?

Question

In the Linux kernel, arch_spin_lock() is implemented as follows:

static inline void arch_spin_lock(arch_spinlock_t *lock)
{
    unsigned int tmp;
    arch_spinlock_t lockval, newval;

    asm volatile(
    /* Atomically increment the next ticket. */
"   prfm    pstl1strm, %3\n"
"1: ldaxr   %w0, %3\n"
"   add %w1, %w0, %w5\n"
"   stxr    %w2, %w1, %3\n"
"   cbnz    %w2, 1b\n"
    /* Did we get the lock? */
"   eor %w1, %w0, %w0, ror #16\n"
"   cbz %w1, 3f\n"
    /*
     * No: spin on the owner. Send a local event to avoid missing an
     * unlock before the exclusive load.
     */
"   sevl\n"
"2: wfe\n"
"   ldaxrh  %w2, %4\n"
"   eor %w1, %w2, %w0, lsr #16\n"
"   cbnz    %w1, 2b\n"
    /* We got the lock. Critical section starts here. */
"3:"
    : "=&r" (lockval), "=&r" (newval), "=&r" (tmp), "+Q" (*lock)
    : "Q" (lock->owner), "I" (1 << TICKET_SHIFT)
    : "memory");
}

Notice the 'wfe' instruction puts the processor in low power mode and waits for the event register to be set. ARMv8 manual specifies that an event is generated if the global monitor for the PE is cleared (section D1.17.1). This should be done by the unlock part. But lets look at the arch_spin_unlock() part:

static inline void arch_spin_unlock(arch_spinlock_t *lock)
{
    asm volatile(
"   stlrh   %w1, %0\n"
    : "=Q" (lock->owner)
    : "r" (lock->owner + 1)
    : "memory");
}

There is no SEV!! So, what is waking up the lock WFE here?

PS: I've been looking for any ARM64 assembly tutorials but nothing came up. Would be awesome if someone has any suggestions. Thanks!

Answer 1

When locking, the line

" ldaxrh %w2, %4\n"

after the wfe performs an exclusive load-acquire of the lock. As stated in the previous comment, this will mark the address of the lock with the global monitor.

The unlock code performs a store-release on the same address

" stlrh %w1, %0\n"

This will generate the event. That is the reason why they use a load-acquire for the lock in the locking function, as opposed to regular load, and why you don't need a SEV when unlocking.

Answer 2

The global monitor event is cleared even when a store occurs to the memory location which has been marked exclusive.

See the state transition diagram B2-5. A store to the marked address will clear the global monitor event because of which an event is generated. This will cause the lock wfe to finish.