Why it fails to load the referenceImage?

Question

Currently, I try to test the ImageTrackingProvider with the Apple Vision Pro. I started with some basic code:

import Foundation
import RealityKit
import ARKit

@MainActor class ARKitViewModel: ObservableObject{
    private let session = ARKitSession()
    private let imageTracking = ImageTrackingProvider(referenceImages: ReferenceImage.loadReferenceImages(inGroupNamed: "AR"))
    
    func runSession() async {
        do{
            try await session.run([imageTracking])
        }
        catch{
            print(error)
        }
    }
    
    func processUpdates() async {
        for await _ in imageTracking.anchorUpdates{
            print("test")
        }
    }
}

I only have one picture in the AR folder. I added the size an I have no error messages in the AR folder.

As I am trying to run the application with the vision Pro, I receive following error:

ar_image_tracking_provider_t <0x28398f1e0>: Failed to load reference image <ARReferenceImage: 0x28368f120 name="IMG_1640" physicalSize=(1.350, 2.149)> with error: Failed to add reference image.

It finds the image, but there seems to be a problem with the loading. I tried the jpeg and the png format.

I do not understand why it fails to load the ReferenceImage.

I use Xcode Version 15.3 beta 3

I access the class ARKitViewModel via the Immersive View:

struct ImmersiveView: View {
    @StateObject var arKitVM = ARKitViewModel()
    var body: some View {
        RealityView { content in
            
        }
        .task {
            await arKitVM.runSession()
        }
        .task {
            await arKitVM.processUpdates()
        }
    }
}

Answer 1

The error message seems to be a know issue in visionsos 1.0. Nevertheless, it works if you are ignoring the error message.

Answer 2

Your reference image has incorrect physicalSize=(1.350, 2.149) because you used comma , separator instead of dot . separator. You've probably changed the decimal separator in macOS settings and it probably doesn't match the ARKit / RealityKit metric system. In the Xcode Inspector, you need to specify the physical size of the image as 1.35 x 2.15 meters. Also, the minimum image size of your JPEG or PNG must be at least 480x480 pixels.

Moreover, since you're running a session with an image tracking provider, you need to use a NSWorldSensingUsageDescription key (set it in info.plist tab) to indicate that your app needs access to the world-sensing data.