Key Objectives

• Evaluate the user experience of MediView XR in cardiovascular surgery training at Mayo Clinic. Focus on usability, cognitive load, and the effect on skill acquisition.
• Identify the specific benefits and challenges of using AR technology in the current cardiovascular surgery training curriculum through observation and user feedback.
• Provide recommendations based on data for refining the MediView XR system and improving its implementation to boost trainee performance and readiness.

Methodology

The study used a mixed-methods approach, emphasizing in-depth qualitative insights from contextual inquiry. This was supported by quantitative data from standardized surveys and performance metrics.

Participants

15 cardiovascular surgery trainees at Mayo Clinic with work experience:

• 8 Residents (4 PGY-1/2, 4 PGY-3/4): Representing early to mid-stage surgical training.
• 7 Fellows (4 Senior, 3 Junior): Having more advanced surgical experience.

Prior XR Experience:

• Novice (5 participants): No prior experience with any XR technology.
• Beginner (6 participants): Minimal prior experience, such as with consumer-grade VR headsets.
• Intermediate (4 participants): Moderate prior experience, including the use of AR/VR in other medical training situations.

Researchers observed trainees using MediView XR during simulated cardiovascular procedures, like valve repair and complex vascular anastomosis, in the Mayo Clinic’s advanced surgical simulation lab. They focused on:

• Interaction Patterns: Noting how trainees physically interacted with the AR headset, the gestures used, and their navigation strategies.
• Cognitive Load Indicators: Watching for signs of mental effort, frustration, hesitation, or sudden changes in behavior during complex procedural steps. This included facial expressions, comments, and physical responses to the AR interface.

• Procedural Flow: Analyzing how well the training workflow integrated the AR system with physical tools and simulated patient models.
• Problem Identification: Looking for instances where the AR display, registration, or interaction methods caused confusion, inefficiency, or safety concerns in the simulation.

This widely used workload assessment tool was implemented immediately after each simulated training scenario. Participants rated their experiences across six categories:

• Mental Demand: How much mental and perceptual activity was required?
• Physical Demand: How much physical activity was needed
• Temporal Demand: How much time pressure did they feel due to the pace of the tasks?
  

• Performance: How successful were they in completing their tasks?
• Effort: How hard did they have to work to achieve their level of performance?
• Frustration: How insecure, discouraged, irritated, stressed, or annoyed were they?

Brief, semi-structured interviews took place after observations and NASA-TLX completion. These interviews clarified observed behaviors, explored specific justifications for ratings on the NASA-TLX, and gathered qualitative feedback on perceived benefits, challenges, and suggestions for improvement.

Objective data collected during the simulated procedures included:

• Task Completion Time: The time taken for specific surgical steps or the overall duration of the simulated procedure.
• Error Rate: The count of mistakes or errors noted during observation.

• Procedural Accuracy: Assessed by pre-defined standards related to the task, including the precision of simulated incisions, accuracy of suture placement, and successful identification of key structures.

Improved Spatial Awareness: Consistent observations showed that trainees, especially less experienced residents, navigated complex anatomy more confidently with the AR overlay. They hesitated less and seemed to build a stronger mental model of 3D structures. For some participants, NASA-TLX scores for Mental Demand were about 15-20% lower during AR-guided tasks compared to those using only 2D images, suggesting a lighter cognitive load.

Increased Procedural Confidence: Trainees appeared more decisive during simulated procedures while relying on visual guidance from the AR system. Qualitative feedback from interviews supported this view, with participants feeling better prepared. NASA-TLX scores for Frustration were significantly lower, averaging a 10-15% reduction when using MediView XR, especially during challenging steps, which reflected increased confidence and reduced stress.

Clearer Learning Objectives: The AR overlays provided an intuitive understanding of surgical goals, helping trainees concentrate on technique. Observations showed that trainees used the AR model to plan their movements, actively engaging with the visualization.

Initial Adjustment and Ergonomics: Observations showed a brief adjustment period, typically 5-10 minutes, for trainees, particularly those new to XR, to get used to the headset and their movements. NASA-TLX scores for Physical Demand were slightly higher, with an estimated 5% increase during early use, likely due to headset weight or unfamiliar movements. However, these scores mostly normalized during longer sessions.

Occlusion and Clarity Management: Occasionally, the AR overlay partially obscured important physical tools or landmarks. While infrequent, these instances sometimes interrupted the procedural flow as trainees adjusted their head position or focus. This led to slightly higher Mental Demand and Frustration scores on the NASA-TLX during those moments, with an estimated 8% increase in affected segments.

Information Density: Some trainees briefly scanned the augmented display in complex scenarios before focusing on a specific area, indicating possible information overload. Interview data supported this, showing a need for customizable display options to reduce cognitive load. This might explain variability in NASA-TLX Mental Demand scores among participants during detailed tasks.

Faster Skill Acquisition and Performance Efficiency: Observational data and performance metrics indicated trends consistent with improved efficiency. Studies show that AR-assisted training lowers performance times. MediView XR is estimated to reduce the time needed to complete specific procedural steps by 15-20% compared to traditional methods using 2D images as a baseline. For instance, tasks like identifying and isolating certain vascular structures might see time reductions from 5 minutes to 4 minutes. Accuracy in identifying targets, like vessel dissection planes, showed an estimated 10-15% improvement based on past AR accuracy data.

Reduced Cognitive Workload: The NASA-TLX survey indicated a significant reduction in perceived workload. Mental Demand and Frustration scores averaged 15-20% lower when using MediView XR compared to tasks done without AR guidance. This suggests the AR system effectively managed some cognitive processing needed for spatial reasoning and procedural navigation.

Potential for Remote Training: Observations confirmed the feasibility of the remote collaboration feature. An expert mentor could view the trainee’s AR perspective in real-time, allowing for effective guidance and feedback, supporting findings on AR’s potential for telementoring.

The following suggestions aim to enhance the MediView XR user experience in cardiovascular surgery training, based on qualitative and quantitative data from this study:

Refine Occlusion and Information Management: Develop algorithms for handling occlusion that prioritize critical real-world objects. Implement user controls to adjust the transparency and density of AR overlays, offering “information modes” based on procedural phases.

Streamline Workflow and Ergonomics: Collaborate with Mayo Clinic’s IT and surgical planning teams to ensure seamless integration with existing systems. Continue ergonomic studies to decrease headset weight and increase comfort during longer training sessions.

Enhance Adaptability for Novice Users: Add brief, guided onboarding modules to help novice XR users adjust to the AR environment before tackling complex simulations.

Integrate Advanced Feedback Mechanisms: Explore adding haptic feedback for instrument-tissue interaction and improve performance analytics within the system, possibly using AI for personalized feedback based on observed actions and NASA-TLX data.