How to prevent panic in Go while stack expands and encounters invalid address in unsafe.Pointer?

Question

I am developing my own interpreter for my custom programming languages that has a couple of types (integers, strings, arrays, functions...). So, I have a dilemma how to represent those types efficiently.

My first option is to create common interface:

type Obj interface {
    Type() Type
    Clone() Obj
    Equals(other Obj) bool
    fmt.Stringer
}

and implement multiple structs that represent each type:

type Int int
func (integer Int) Type() types.Type { return types.TypeInt }
func (integer Int) Clone() types.Obj { return integer }
func (integer Int) Equals(other types. Obj) bool { ... }

...

type Bool bool
func (boolean Bool) Type() types.Type { return types.TypeBool }
func (boolean Bool) Clone() types.Obj { return boolean }
func (boolean Bool) String() string   { return strconv.FormatBool(bool(Boolean)) }

...

type Array struct {
    Slice []types. Obj
}
func (array *Array) Type() types.Type { return types.TypeArray }
func (array *Array) Clone() types.Obj { ... }
func (array *Array) Equals(other types. Obj) bool { ... }

So instances of variables in the interpreted language would be stored as instance of interface.

Everything works correctly, but that approach is slow. Reason for slowness is dynamic casting (from the interface to a concrete struct type) and a lot of heap allocations.

Then an idea of tagged union came to my mind. But, Golang doesn't have unions, so I needed to emulate them. I created a struct with tag and array of bytes:

type Tag int

type Object struct {
    Tag  Tag
    Data [MAX_SIZE]byte
}

So after inspecting tag, I would reinterpret data from array as a concrete type:

func As[T any](obj Obj) T {
    ptr := (*T)unsafe.Pointer(&obj.Data[0])
    return *ptr
}

I re-implemented everything to work with this type system. But things started to work weirdly. Latter I recognized that for storing some data types (like strings and arrays) pointers are needed, and in this representation, they will be treated as any other numeric data in that byte array, so GC would delete those objects and maybe allocate some new objects in that same place.

So, to give "hint" to GC not to do that, I did some researching and found out that GC cares about unsafe.Pointer and will check if pointer is valid and correctly behave.

So I changed my emulated union struct to be:

type Tag int

type Object struct {
    Tag  Tag
    Data [MaxTypeSize / unsafe.Sizeof(unsafe.Pointer(nil))]unsafe.Pointer
}

And everything worked correctly sometimes, but sometimes didn't. Program was panicking and giving me

runtime: bad pointer in frame banek/interpreter.(*interpreter).evalBinaryOp at 0xc000292890: 0x1
fatal error: invalid pointer found on stack

I followed stack trace and found out that it panicked from function that expands the stacks and does pointer reallocations. And at that moment, of stack expansion, some instance of my struct contained non-pointer data (probably just an integer) in array of unsafe.Pointer.

So my question is: Is there some way to stop the function from adjusting pointers at all, but that GC still continues to detect them (so can't use uintptr). It would be safe, because I won't have pointers to stack values, and adjusting won't be needed.

Or, if that is not possible, can I emulate an union in some other way, but without manual reference counting and pinning objects.