What is Non-Maximum Suppression?

The aim of object detection is to detect and label objects in a image or video which is done by drawing rectangular boxes around the objects. These boxes indicate the position of the object in the image and Non-Maximum Suppression (NMS) is a concept used there and in this article we will learn more about it.

Understanding Non-Maximum Suppression (NMS)

Non-Maximum Suppression (NMS) is a method used in object detection to remove extra boxes that are detected around the same object. When an object is detected multiple times with different bounding boxes, NMS keeps the best one and removes the rest. This helps us to make sure each object is counted only once, improving the accuracy and clarity of the results. The role of NMS is to:

• Reduce Redundancy: By removing overlapping boxes, NMS ensures that a single, high-confidence bounding box represents each object.
• Improve Precision: By retaining the bounding box with the highest confidence score, NMS enhances the accuracy of the detection results. It reduces number of false positives.
• Computational Efficiency: It improves the computational efficiency of the object detection process by reducing number of boxes to a manageable and relevant set.

Here we will see the ,step-by-step working of NMS:

1. Bounding Box Generation

The object detection model generates multiple bounding boxes for objects in an image each with a confidence score indicating the likelihood of the presence of an object.

2. Sorting by Confidence Score

All the generated bounding boxes are sorted in descending order based on their confidence scores.

3. Suppression Process

1. Select the Box with the Highest Confidence: The box with the highest confidence score is considered a final detection.
2. Calculate Intersection over Union (IoU): IoU is a measure of the overlap between two bounding boxes.
3. Suppress Overlapping Boxes: If the IoU of any remaining boxes with the selected box exceeds a threshold (usually 0.5) they are suppressed and discarded.
4. Repeat the Process: This continues until all boxes have been either selected as a detection or suppressed.

4. Output the Final Detections

The remaining boxes after the suppression process are output as the final detection results each representing a unique object in the image with high confidence.

Example Workflow

Let's consider an image where multiple bounding boxes are generated around a single object in this case a car:

1. Initial Detections: The model generates several bounding boxes around the car, each with varying confidence scores.
2. Sort by Confidence: Boxes are sorted and the box with the highest confidence score is selected first.
3. Calculate IoU: IoU is calculated for this box against all other boxes. Suppose the selected box has an IoU of 0.7 with a nearby box.
4. Suppress Boxes: Since 0.7 is above the threshold of 0.5 the nearby box is suppressed.
5. Repeat: The next highest confidence box is selected and the process repeats until all boxes are either selected or suppressed.

Types of Non-Maximum Suppression

1. Greedy NMS

Greedy Non-Maximum Suppression is the most commonly used variant of NMS. It follows a straightforward approach to suppress redundant bounding boxes based on their confidence scores and their overlap with each other.

2. Soft NMS

Soft Non-Maximum Suppression addresses some of the shortcomings of Greedy NMS by reducing the confidence scores of overlapping bounding boxes instead of completely suppressing them. This approach allows for a more better suppression mechanism.

Process:

1. Select the bounding box with the highest confidence score as a detection and calculate IoU between this box and all other remaining boxes.
2. Instead of discarding boxes with high IoU, reduce their confidence scores according to a decay function like linear or Gaussian decay.
3. Repeat the process for the next highest confidence box from the remaining boxes until all boxes have been processed.
4. After updating the confidence scores apply a threshold to remove boxes with scores below a certain value.
5. The remaining boxes are the final detections.

Differences between Soft NMS and Greedy NMS:

• Score Reduction vs Suppression: Soft NMS reduces the confidence scores of overlapping boxes instead of outright discarding them.
• Suppression: The suppression in Soft NMS is more gradual, allowing for a softer reduction in the number of detections.

NMS is used in object detection to refine and improve quality of the final detected objects. It ensures that each object is represented by a single, precise bounding box, enhancing the accuracy and efficiency of the object detection system.