Can Synthetic Data Improve Multi-Class Counting of Surgical Instruments?

Counting is a common preventative measure taken to ensure surgical instruments are not retained during surgery, which could cause serious detrimental effects including chronic pain and sepsis. A hybrid human-AI system could support or partially automate this manual counting of instruments. An important element to evaluate the viability of using deep learning computer vision-based counting is a suitable large-scale dataset of surgical instruments. Other domains, such as crowd analysis and instance counting, have leveraged synthetic datasets to evaluate and augment different approaches. We present a synthetic dataset (SORT), which is complemented by a smaller real-world dataset of surgical instruments (MSMI), to assess the hypothesis whether synthetic training data can improve the performance of multiclass multi-instance counting models when applied to real-world data. In this preliminary study, we provide comparative baselines for various popular counting techniques on synthetic data, such as direct regression, segmentation, localisation, and density estimation. These experiments are repeated at different resolutions - full high-definition (1080 × 1920 pixels), half (690 × 540 pixels), and quarter (480 × 270 pixels) - to measure the robustness of different supervision methods to varying image scales. The results indicate that neither the degree of supervision nor the image resolution during model training impact performance significantly on the synthetic data. However, when testing on the real-world instrument dataset, the models trained on synthetic data were significantly less accurate. These results indicate a need for further work in either the refinement of the synthetic depictions or fine-tuning on real-world data to achieve similar performance in domain adaptation scenarios compared to training and testing solely on the synthetic data.