ICRoM 2022 Grand Challenge

Overview Important Dates Challenge Description Submission Instructions Registration Fees FAQ

Very Dear Participants,

We thank you for signing up for ICRoM 2022 Grand Challenge, but we regret to inform you that the challenge will not be held this year, since the number of eligible participating teams did not reach the minimum required number for a competitive event. We hope to be able to perform the challenge, next year with your support. Your registration fee will be refunded by the conference’s financial team within two weeks.
Thanks for your kind contribution to ICRoM 2022 conference, we wish to see you in this great event.

Regards,
ICRoM Challenge Executive Team

Overview

The ability to automatically detect and track the surgical main areas in cataract surgery video will enable many transformational interventions. There are a number of applications that would benefit from this technology, including assistive interventions in teaching and performing surgery, and planning operational and logistical aspects of OR resources. However, obtaining the annotations needed to train AI models to identify and localize the surgical main areas is very challenging.

In this challenge, we invite the data scientist and researchers to leverage an annotated dataset as the input to train their machine learning models in order to detect and localize two main areas in capsulorhexis surgery with bounding boxes in the video frames. In simple terms, our problem is object tracking, where objects are the two main areas of surgery. The figure below shows these two areas. Once the challenge begins, we will provide you with a part of the ARFaT dataset, which includes images of video frames and related annotations. You will be instructed on how to upload your model’s results, as well as how ICRoM organizers will evaluate them. You are free to use any model you like, and any approach you find suitable.

Get ready to register and to join us in this challenge, to win the valuable challenge prize (120,000,000 IRR)!

Important dates

Pre-Registration: 8-14th October, 2022
Registration open: 15th October – 1st November, 2022
Release of detailed submission instructions and training data: 2nd-5th October, 2022
End of testing phase: 12th November, 2022
Methodology reports due: 13th November, 2022
Leaderboard finalized and winners announced: 17th November, 2022
Challenge Day: 17th November, 2022
Please note that these dates are subject to change.

Challenge Description

The dataset consists of video clips taken from the surgical microscope camera of Farabi ophthalmology hospital.
This surgical video shows surgeons removing a layer from the eye’s surface and continuing it in a continuous circle-like pattern.
There are 20 video clips, captured at ~25 fps with a resolution of 540p (540 x 540). You can see example screenshots below.

Each video can include two main areas of surgery that are important to us. The image below shows these areas.

We have prepared classes and locations for the two main areas of the surgery on all the videos in the train set. In addition, you will be given more unlabeled video data to label if more data is needed.
We will provide you with the image frames of each video along with the label of these image frames for each video data. Below is an example of these frames and their labels.

You can change the labels using different formats, but note that the final evaluation will be done on the video data with a similar labeling format to the above.
Evaluation of the submissions will be performed on a private test data set containing individual files of surgical videos through grand-challenge automated docker submission and evaluation system. The test data and labels are hidden from the participants. The specific metrics to be used for evaluation is: COCO mAP@[0.5:0.05:0.95] (https://cocodataset.org/#detection-eval)

Submission Instructions

1: Train your desired model using the labeled training dataset we provide. You are free to use existing or modified models and divide the data into train and validation.

2 (Initial Tests): You will receive 20-30% of the test data to run your tracking algorithm. You are provided with the locations of the bounding boxes in the first frame of the test data. As an output of your algorithm, you should generate text files similar to the labels in the ARFaT dataset. We evaluate the tracking algorithm’s performance after you email us the output of your tracking algorithm. This step aims to evaluate your performance and improve your algorithm if necessary.

3 (Final Tests): You will receive the rest of the test data, and the initial test process will be repeated every night. The winners will be determined at the end of this part.

4: The winning teams will present their algorithms and prepare a presentation in a format that will be announced to determine the final winner. As part of the evaluation process, we consider three criteria: accuracy in the centers and overlap areas of the bounding boxes, as well as the failure rate of the tracking algorithm.

The initial and final tests may be combined depending on the time constraints. You can follow this page for updates until October 25th.

Grand Challenge Registration Fees

CATEGORY	BEFORE Oct. 22	AFTER Oct. 22
International Participants	€ 100	€ 150
Iranian Affiliated Participants	IRR 4,000,000	IRR 5,000,000

20% discount for IEEE members.

FAQ

In the final article of the challenge, whose names will be included? The main article will include only the names of the authors and those who directly contributed to preparing the data and producing the results.
Is there going to be more than one winning team in the challenge? Only one team will win the prize. A team that has achieved the best results and can present them effectively.
What is the cost of registering for the challenge? Is it individual or group? This page specifies the registration fee in the section “Registration Fee”. The fee will be paid in groups of one to three people (Only the team leader needs to perform the final registration step).

If the number of participating teams does not reach 10-15 groups, this year's challenge will not be held, and the prize will be transferred to the next year.

Overview

Important dates

Challenge Description

Submission Instructions

Grand Challenge Registration Fees

International Participants

Iranian Affiliated Participants

FAQ