XR Latency Optimization

System Architecture

Key Implementation

// Future Timestamp Execution Strategy
public void SchedulePlayback(long targetServerTime) {
    // 1. Calculate precise local offset using RTT
    // timeOffset = ServerTime - LocalTime - (RTT / 2)
    long localTargetTime = targetServerTime - this.timeOffset;
    
    // 2. Calculate exact wait duration
    // WaitTime = TargetTime - CurrentLocalTime
    double waitSeconds = (localTargetTime - DateTime.UtcNow.Ticks) / 10000000.0;
    
    if (waitSeconds > 0) {
        // 3. Schedule execution
        StartCoroutine(ExecuteAtTime(waitSeconds, () => {
            videoPlayer.Play(); // Triggers at the exact same physical moment
        }));
    } else {
        // Fallback: Seek to correct position if packet arrived late
        videoPlayer.time += Math.Abs(waitSeconds);
        videoPlayer.Play();
    }
}

Technical Trade-offs

Initially, I used socket.broadcast.emit to sync time, but this flooded the main thread of client devices (O(n^2) message complexity), causing stuttering. I refactored the architecture to use Targeted Emits (io.to(guestId).emit). This reduced CPU overhead by selectively sending correction data only to clients that drifted beyond the ±3 frame threshold, prioritizing network stability over constant updates.

Error Handling Strategy

Developed a real-time CSV logging system to record CurrentFrame from all clients during stress tests. Implemented a Fail-Safe LOD (Level of Detail) system where devices exceeding 150ms RTT automatically downgrade video resolution to maintain synchronization priority. Disconnected clients can hot-swap back into the session using the calculated global time offset without restarting the experience.