For many surgeons, the possibility of going back into the operating room to review the actions they carried out on a patient could provide invaluable medical insights.
Using a mix of Facebook's PyTorch framework and machine-learning platform Allegro Trains, med-tech company theator is now providing surgeons with a tool that lets them watch over and analyze in detail the past operations they have performed, and access video footage of procedures carried out by colleagues around the world.
Dubbed the "surgical intelligence platform", theator's platform uses computer vision technology to extract key information from videos taken during surgical operations. The data is annotated, compiled and organized to let doctors review specific content by simply typing in key words through the platform. Surgeons can use the tool to jump to a specific step, re-watch critical moments, or access analysis about the procedure, such as time taken to perform a given action.
theator's data scientists use a number of sophisticated machine-learning models to index and catalogue the video content provided by surgeons – but they quickly realized, as they were faced with mounting content, that manually training the models was a task more challenging than anticipated.
"We realized that running all of these processes manually was infeasible and automating training pipelines was an absolute must," said Omri Bar, research team lead at theator in a new blog post. "Now, when new data comes in, it's immediately processed and fed directly into training pipelines – speeding up workflow, minimizing human error, and freeing up our research team for more important tasks."
Machine-learning pipelines consist of many different processes including data collection, preparation and cleaning, feature extraction or model validation. These tasks are traditionally completed by data scientists, which is more costly and can take up to several months.
Automating some of the more