'IndexOutOfBoundsException: Out of bound access on segment ...' when accessing pointer read from MemorySegment

Question

I am attempting to test the Foreign Function and Memory features of Java 21. Here is my code :

public static void main(String[] args) {
        // 1. Find foreign function on the C library path
        Linker linker = Linker.nativeLinker();
        SymbolLookup stdlib = linker.defaultLookup();
        MethodHandle radixsort = linker.downcallHandle(stdlib.find("radixsort").orElseThrow(), FunctionDescriptor.ofVoid(ValueLayout.ADDRESS, ValueLayout.JAVA_INT, ValueLayout.ADDRESS, ValueLayout.JAVA_CHAR));
        // 2. Allocate on-heap memory to store four strings
        String[] javaStrings = {"mouse", "cat", "dog", "car"};

        // 3. Use try-with-resources to manage the lifetime of off-heap memory
        try (Arena offHeap = Arena.ofConfined()) {
            // 4. Allocate a region of off-heap memory to store four pointers
            MemorySegment pointers = offHeap.allocateArray(ValueLayout.ADDRESS, javaStrings.length);
            // 5. Copy the strings from on-heap to off-heap
            for (int i = 0; i < javaStrings.length; i++) {
                MemorySegment cString = offHeap.allocateUtf8String(javaStrings[i]);
                pointers.setAtIndex(ValueLayout.ADDRESS, i, cString);
                assert cString.getUtf8String(0).length() > 0;
            }
            // 6. Sort the off-heap data by calling the foreign function
            radixsort.invoke(pointers, javaStrings.length, MemorySegment.NULL, '\0');
            // 7. Copy the (reordered) strings from off-heap to on-heap
            for (int i = 0; i < javaStrings.length; i++) {
                MemorySegment cString = pointers.getAtIndex(ValueLayout.ADDRESS, i);
                javaStrings[i] = cString.getUtf8String(0);
            }
        } // 8. All off-heap memory is deallocated here
        catch (Throwable e) {
            throw new RuntimeException(e);
        }
        assert Arrays.equals(javaStrings, new String[]{"car", "c∞at", "dog", "mouse"});  // true
    }

but I am encountering this error :

Exception in thread "main" java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Out of bound access on segment MemorySegment{ heapBase: Optional.empty address:105553153094096 limit: 0 }; new offset = 0; new length = 1

At the line :

javaStrings\[i\] = cString.getUtf8String(0); It seems that MemorySegment cString = pointers.getAtIndex(ValueLayout.ADDRESS, i); returns a MemorySegment with a byteSize = 0

When I use cString.getUtf8String(0); on the original cString, I do not encounter an error.

It seems that pointers.setAtIndex and pointers.getAtIndex do not preserve the MemorySegment byteSize. ?

Answer 1

It seems that pointers.setAtIndex and pointers.getAtIndex do not preserve the MemorySegment byteSize. ?

Yes, that is correct. The address is stored into off-heap memory as a plain memory address. The size is dropped. So, when reading an address back from off-heap memory, the created memory segment has size 0.

This is explained in the javadoc of MemorySegment as well:

When interacting with foreign functions, it is common for those functions to allocate a region of memory and return a pointer to that region. Modeling the region of memory with a memory segment is challenging because the Java runtime has no insight into the size of the region. Only the address of the start of the region, stored in the pointer, is available.

...

The MemorySegment API uses zero-length memory segments to represent:

• pointers returned from a foreign function;
• pointers passed by a foreign function to an upcall stub; and
• pointers read from a memory segment (more on that below).

Your use case is the latter of the three.

You have to use MemorySegment::reinterpret to resize the segment explicitly. Since for a C string the actual size is not known (it's indicated by a null terminator), you can resize the segment to Long.MAX_VALUE to make it accessible:

// 7. Copy the (reordered) strings from off-heap to on-heap
for (int i = 0; i < javaStrings.length; i++) {
    MemorySegment cString = pointers.getAtIndex(ValueLayout.ADDRESS, i);
    cString = cString.reinterpret(Long.MAX_VALUE); // <----
    javaStrings[i] = cString.getUtf8String(0);
}

---

Alternatively, you can create an AddressLayout with a target layout that has the needed size, to avoid the separate call to reinterpret:

AddressLayout UNBOUNDED_POINTER = ValueLayout.ADDRESS
        .withTargetLayout(MemoryLayout.sequenceLayout(Long.MAX_VALUE, ValueLayout.JAVA_BYTE));

And then use that to do the dereference:

// 7. Copy the (reordered) strings from off-heap to on-heap
for (int i = 0; i < javaStrings.length; i++) {
    MemorySegment cString = pointers.getAtIndex(UNBOUNDED_POINTER, i);
    // no reinterpret needed
    javaStrings[i] = cString.getUtf8String(0);
}