Hand function is central to independence and quality of life, yet current rehabilitation and clinical assessments rarely capture the complexity and variability of real-world hand use. Researchers at UHN’s KITE Research Institute are using AI to assess hand movements in individuals with spinal cord injury—an approach that could lead to tailored rehabilitation strategies with real-world applications.
The research team analyzed hand movements in 19 individuals with cervical spinal cord injuries using home-based video recordings. Participants wore small cameras while performing everyday tasks like cooking, cleaning, or getting dressed. These point-of-view videos showed how people used their hands in natural settings. AI was then used to study the footage and group similar hand movements together.
Unlike traditional assessments conducted in controlled clinical settings with standardized objects, this approach used deep learning models—a type of AI capable of recognizing complex patterns—to identify each person’s unique grasping styles. The AI looked at both posture data (how the hand is shaped) and appearance data (what the hand is interacting with) to identify common grasping patterns. The model was able to group similar movements with 68% accuracy, showing how people adapt their hand use after injury.
This personalized analysis provides clinicians and rehabilitation specialists a better understanding of how individuals adjust their hand movements in daily life. It could help design rehabilitation programs and technologies that focus on hand function used at home. This method may also be useful in fields such as robotics, sports science, and ergonomics, where understanding hand movement is essential.
Dr. Mehdy Dousty, first author of the study, was a former PhD candidate in the labs of Drs. Jose Zariffa and David Fleet.
Dr. Jose Zariffa, senior author of the study, is a Senior Scientist at UHN’s KITE Research Institute (KITE) and holds the KITE Chair in Spinal Cord Injury Research. At the University of Toronto, Dr. Zariffa is an Associate Professor in the Institute of Biomedical Engineering, a Faculty Member of the Rehabilitation Sciences Institute, and a Cross-Appointed Professor in the Edward S. Rogers Sr. Department of Electrical & Computer Engineering.
This work was supported by the Natural Sciences and Engineering Research Council of Canada, Praxis Spinal Cord Institute, the Ontario Ministry of Colleges, Universities, Research Excellence and Security, Craig H. Neilsen Foundation, Healthcare Robotics (HeRo), the NVIDIA Corporation, and UHN Foundation.
Dousty M, Fleet DJ, Zariffa J. Personalized Video-Based Hand Taxonomy Using Egocentric Video in the Wild. IEEE J Biomed Health Inform. 2025 Sep. doi: 10.1109/JBHI.2024.3495699.