What You Should Know:
– Mount Sinai researchers have demonstrated the effectiveness of teaching surgical trainees a difficult procedure using artificial intelligence (AI) algorithms and an extended-reality headset without the presence of an instructor. All of the 17 trainees in the study achieved surgical success.
– The novel study, published in Journal of Medical Extended Reality, drew highly favorable reviews from student participants who tested the deep learning model. The results carry significant implications for future training of residents and surgeons, as well as for the even broader field of autonomous learning within medicine.
Mount Sinai Develops AI-Driven Extended Reality Surgical Training with 99.9% Accuracy
Pioneering AI-Enhanced Surgical Education
Mount Sinai Health System—one of the largest academic medical systems in the New York metro area—operates seven hospitals, more than 400 outpatient practices, over 600 research and clinical labs, a nursing school, and the Icahn School of Medicine. With 48,000 employees and approximately 9,000 primary and specialty care physicians, the system delivers comprehensive care from birth through geriatrics, integrating artificial intelligence, informatics, and patient-centered approaches across New York City, Westchester, Long Island, and Florida.
In a groundbreaking study, Mount Sinai researchers created an AI model paired with a custom extended-reality headset to demonstrate that a critical step in a kidney cancer procedure could be performed with 99.9% accuracy. “Our study offers early proof that AI programs substituting for live proctors can reduce training costs while improving quality, efficiency, and standardization,” said Nelson Stone, MD, Clinical Professor of Urology, Radiation Oncology, and Oncological Sciences at the Icahn School of Medicine and corresponding author of the study.
Developed in collaboration with the University of Rochester Medical Center’s Department of Neurosurgery, the system—called ESIST (Educational System for Instructionless Surgical Training)—uses deep learning and a headset to stream step-by-step surgical instructions and videos directly into the trainee’s field of vision. This allows residents to keep their hands free to practice intricate procedures while receiving real-time corrective prompts.
Simulated Kidney Cancer Surgery and Future Applications
The pilot focused on a simulated partial nephrectomy—removing a cancerous kidney segment and clamping the renal artery. Researchers used anonymized CT scans to 3D print a “phantom” kidney, filling it with water-based polymers to mimic tissue and tumor consistency. Seventeen participants practiced the procedure while the AI system monitored performance via a first-person camera, delivering instant feedback to refine technique.
Encouragingly, 100% of participants reported the program had significant educational value. The Mount Sinai team now aims to extend the AI algorithm to develop more complex synthetic cadaver models, enabling full-procedure training rather than focusing on isolated surgical steps.
Authors of the study, as listed in the journal, include Jonathan J. Stone, Nelson N. Stone, Steven H. Griffith, Kyle Zeller, and Michael P. Wilson.
By merging advanced AI with extended reality, Mount Sinai is reimagining surgical training—making it more consistent, cost-effective, and scalable for the next generation of surgeons.
