Skip to main content

Automated tool detection with deep learning for monitoring kinematics and eye hand coordination in microsurgery

Research Abstract

In microsurgical procedures, surgeons use micro-instruments under high magnifications to handle delicate tissues. These procedures require highly skilled attentional and motor control for planning and implementing eye-hand coordination strategies. Eye-hand coordination in surgery has mostly been studied in open, laparoscopic, and robot-assisted surgeries, as there are no available tools to perform automatic tool detection in microsurgery. We introduce and investigate a method for simultaneous detection and processing of micro-instruments and gaze during microsurgery. We train and evaluate a convolutional neural network for detecting 17 microsurgical tools with a dataset of 7500 frames from 20 videos of simulated and real surgical procedures. Model evaluations result in mean average precision at the 0.5 threshold of 89.5–91.4% for validation and 69.7–73.2% for testing over partially unseen surgical techniques.

Research Authors
Jani Koskinen, Mastaneh Torkamani-Azar, Ahmed Hussein, Antti Huotarinen, Roman Bednarik
Research Date
Research Department
Research Journal
Computers in Biology and Medicine

] Image Synthesis with Generative Adversarial Networks to Augment Tool Detection in Microsurgery

Research Abstract

For the first time in literature, we investigate the capability of Generative Adversarial Networks (GAN) for synthesizing realistic images of microsurgical procedures and augmenting training data for surgical tool detection. We employ videos from practice and intraoperative neurosurgical procedures to train and evaluate two recent GAN models that have shown promise in high-resolution image generation: StyleGAN2 with Adaptive Discriminator Augmentation and StyleGAN2 with Differential Augmentation. Models were trained with limited data for both conditional and unconditional image generation, where the conditional models generated images with and without surgical tools. Our results show that the unconditional models achieved FID scores between 6 and 25 units lower than the conditional models for the two practice datasets. The best performance (FID= 42.16 and 25.17) was achieved in the Go-around practice task and was comparable to the previous benchmark performance of StyleGAN2 with Differential Augmentation. Experts’ visual inspection showed that while synthetic images had faults that exposed their true origin to the human eye, a sizable portion of them included identifiable surgical instruments. Experiments with object detection showed that augmenting the training data with synthetic microsurgical data improved the mean average precision for detecting tool tips in practice microsurgery datasets by 3%. Future work will include improving the quality of image synthesis and investigating key visual cues in expert assessment of surgical scenes for applications in robust surgical tool detection, bimanual skill evaluation

Research Authors
Mastaneh Torkamani-Azar, YuChun Liu, Jani Koskinen, Ahmed Hussein, Matti Iso-Mustajärvi, Hana Vrzakova, Roman Bednarik
Research Date
Research Department

Optimal spectral bands for instrument detection in microscope-assisted surgery

Research Abstract

Optic image-guidance systems enable minimally invasive (MIS) approaches in surgery. However, available MIS-techniques limits both ergonomics and field of view (FoV), which can be detrimental for anatomical awareness and safe manipulation with tissues. Contemporary navigation techniques (i.e. neuronavigation) support spatial awareness during surgery. However, these techniques require time-consuming instrumentation and lack real-time precision needed in soft-tissue surgery. In this work, we utilize operative microscopes FoV as an unobtrusive source to support MIS-navigation with micro-instrument tracking. The FoV instrument tracking has been investigated in laparoscopy, however, high magnification, selection of instruments and bimanually variant characteristics of microneurosurgery make the current computational approaches challenging to adopt. In this work, we investigate potentials of spectral

Research Authors
Sami Puustinen, Jani Koskinen, Hana Vrzakova, Piotr Bartczak, Ahmed Hussein, Samu Lehtonen, Antti-Pekka Elomaa
Research Date
Research Journal
2020 IEEE 33rd International Symposium on Computer-Based Medical Systems (CBMS)

Automated tool detection with deep learning for monitoring kinematics and eye-hand coordination in microsurgery

Research Abstract

In microsurgical procedures, surgeons use micro-instruments under high magnifications to handle delicate tissues. These procedures require highly skilled attentional and motor control for planning and implementing eye-hand coordination strategies. Eye-hand coordination in surgery has mostly been studied in open, laparoscopic, and robot-assisted surgeries, as there are no available tools to perform automatic tool detection in microsurgery. We introduce and investigate a method for simultaneous detection and processing of micro-instruments and gaze during microsurgery. We train and evaluate a convolutional neural network for detecting 17 microsurgical tools with a dataset of 7500 frames from 20 videos of simulated and real surgical procedures. Model evaluations result in mean average precision at the 0.5 threshold of 89.5–91.4% for validation and 69.7–73.2% for testing over partially unseen surgical

Research Authors
Jani Koskinen, Mastaneh Torkamani-Azar, Ahmed Hussein, Antti Huotarinen, Roman Bednarik
Research Date
Research Department
Research Journal
Computers in Biology and Medicine

Airway pressure release ventilation versus pressure-controlled ventilation in acute hypoxemic respiratory failure.

Research Authors
R Ibrahim, Y Mohamed, M Abd El-Kader, A Azouz
Research Date
Research Department
Research Journal
The Egyptian Journal of Chest Diseases and Tuberculosis 71 (1), 74-74
Research Member

Association between periodontitis and cardiovascular health in rheumatoid arthritis patients: Prospective effect of periodontal treatment on cardiovascular risk

Research Abstract

Rheumatoid arthritis (RA) patients are more likely to develop cardiovascular disease (CVD), which increases the risk of morbidity and mortality. Periodontitis is known to be associated with CVD, yet its relationship with CVD in RA is limited. Aim of the work: To examine the relationship between periodontitis with subclinical atherosclerosis and with long term CVD risk. Examining if periodontitis treatment can be associated with CVD improvement was well thought out. Patients and methods: This prospective interventional study included 49 adults with RA. Demographic, clinical and therapeutic data and laboratory markers were assessed. Dental examination for periodontitis was performed. The carotid intima media thickness (CIMT) and Framingham risk score (FRS) were evaluated. Medical treatment was provided to RA patients with periodontitis, and assessments were repeated after 6 months. Results: The mean age of the patients was 46.4 ± 12.4 years, disease duration 10.9 ± 5.4 years and 79.6% were females. 25 (51%) patients had subclinical atherosclerosis, 30 (61.2%) had periodontitis and 25 (51%) had both. RA patients with subclinical atherosclerosis had higher clinical attachment loss (CAL) (3.12 ± 1.45) and higher probing depth (PD) (4.96 ± 1.37) compared to those without (p < 0.001). CAL (b = 0.01, 95 %CI: 0–0.01, p < 0.001), and PD (b = 0.01, 95% CI: 0–0.01; p < 0.001) were independently associated with CIMT. The 30 patients after treatment of periodontitis showed an average improvement in the mean CIMT (0.14 mm, p < 0.001). Conclusion: Periodontitis is associated with subclinical atherosclerosis in RA. Treatment of periodontitis could improve the cardiovascular health in RA patients and prompts physicians to early diagnose and treat periodontitis.

Research Authors
Shaimaa Salah a,⇑ , Nevin Hammam , Mohamed Raouf Abdel Razek , Nadia M. Ismail , Ahmed Mortada Fikry , Wageeh A. Ali , Essam A. Abda
Research Date
Research Journal
The Egyptian Rheumatologist
Research Year
2023

Comparison between central and automated peripheral blood pressure measurement for early detection of kidney dysfunction in hypertensive patients

Research Abstract

There is a close relationship between blood pressure levels and the risk of cardiovascular events, strokes, and kidney disease. For many years, the gold standard instrument for blood pressure measurement was a mercury sphygmomanometer and a stethoscope, but this century-old technique of Riva-Rocci/Korotkov is being progressively removed from clinical practice. Central blood pressure is considered better than peripheral blood pressure in predicting cardiovascular events, as it assesses wave reflections and viscoelastic properties of the arterial wall which make systolic and pulse pressures vary from central to peripheral arteries, but mean blood pressure is constant in the conduit arteries.

Research Authors
Ahmed Abdel-Galeel, Nader N. Fawzy, Wageeh A. Ali & Doaa A. Fouad
Research Date
Research Department
Research Journal
BMC Cardiovascular Disorders
Research Year
2023
Subscribe to