Dunfey · Hotel WWDC as data, est. 1983
Front desk everything
Years
Topics

2024 AI & Machine LearningSpatial Computing

WWDC24 · 17 min · AI & Machine Learning / Spatial Computing

Explore object tracking for visionOS

Find out how you can use object tracking to turn real-world objects into virtual anchors in your visionOS app. Learn how you can build spatial experiences with object tracking from start to finish. Find out how to create a reference object using machine learning in Create ML and attach content relative to your target object in Reality Composer Pro, RealityKit or ARKit APIs.

Watch at developer.apple.com ↗

Transcript all transcripts

Chapters

Code shown on screen · 3 snippets

Coaching UI - display object USDZ preview swift · at 13:55 ↗
// Display object USDZ
struct ImmersiveView: View {
   @State var globeAnchor: Entity? = nil
    var body: some View {
        RealityView { content in
            // Load the reference object with ARKit API
            let refObjURL = 
            Bundle.main.url(forResource: "globe", withExtension: ".referenceobject")
            let refObject = try? await ReferenceObject(from: refObjURL!)

            // Load the model entity with USDZ path extracted from reference object
            let globePreviewEntity = 
            try? await Entity.init(contentsOf: (refObject?.usdzFile)!)

            // Set opacity to 0.5 and add to scene
            globePreviewEntity!.components.set(OpacityComponent(opacity: 0.5))
            content.add(globePreviewEntity!)
        }
    }
}
Coaching UI - check anchor state swift · at 14:13 ↗
// Check anchor state
struct ImmersiveView: View {
   @State var globeAnchor: Entity? = nil
    var body: some View {
        RealityView { content in
            if let scene = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
                globeAnchor = scene.findEntity(named: "GlobeAnchor")
                content.add(scene)
            }
            let updateSub = content.subscribe(to: SceneEvents.AnchoredStateChanged.self) { event in
                if let anchor = globeAnchor, event.anchor == anchor {
                    if event.isAnchored {
                        // Object anchor found, trigger transition animation
                    } else {
                        // Object anchor not found, display coaching UI
                    }
                }
            }
        }
    }
}
Coaching UI - Transform space with SpatialSession swift · at 14:31 ↗
// Transform space
struct ImmersiveView: View {
   @State var globeAnchor: Entity? = nil
    var body: some View {
        RealityView { content in
            // Setup anchor transform space for object and world anchor
            let trackingSession = SpatialTrackingSession()
            let config = SpatialTrackingSession.Configuration(tracking: [.object, .world])
            if let result = await trackingSession.run(config) {
                if result.anchor.contains(.object) {
                    // Tracking not authorized, adjust experience accordingly
                }
            }
           // Get tracked object's world transform, identity if tracking not authorized
            let objectTransform = globeAnchor?.transformMatrix(relativeTo: nil)
            // Implement animation
            ...
        }
    }
}

Resources