目标檢測 - IoU和GIoU作為邊框回歸的損失和代碼實作

flyfish

GIoU

=General-IOU

=Generalized Intersection over Union

論文《Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression》

IoU和GIoU作為邊框回歸的損失

GIoU as Loss for Bounding Box Regression

算法過程如下

輸入: 預測的邊框 B p B^p Bp 和 GT邊框 B g B^g Bg 的坐标

B p = ( x 1 p , y 1 p , x 2 p , y 2 p ) B^p = (x^p_1,y^p_1,x^p_2,y^p_2) Bp=(x1p​,y1p​,x2p​,y2p​), B g = ( x 1 g , y 1 g , x 2 g , y 2 g ) B^g = (x^g_1,y^g_1,x^g_2,y^g_2) Bg=(x1g​,y1g​,x2g​,y2g​)

輸出: L I o U \mathcal{L}_{IoU} LIoU​, L G I o U \mathcal{L}_{GIoU} LGIoU​

1. 對于預測的邊框 B p B^p Bp, 確定 x 2 p > x 1 p x^p_2>x^p_1 x2p​>x1p​ ， y 2 p > y 1 p y^p_2>y^p_1 y2p​>y1p​:

   x ^ 1 p = min ⁡ ( x 1 p , x 2 p ) \hat{x}^p_1 = \min(x^p_1,x^p_2) x^1p​=min(x1p​,x2p​),

   x ^ 2 p = max ⁡ ( x 1 p , x 2 p ) \hat{x}^p_2 = \max(x^p_1,x^p_2) x^2p​=max(x1p​,x2p​),

   y ^ 1 p = min ⁡ ( y 1 p , y 2 p ) \hat{y}^p_1 = \min(y^p_1,y^p_2) y^​1p​=min(y1p​,y2p​),

   y ^ 2 p = max ⁡ ( y 1 p , y 2 p ) \hat{y}^p_2 = \max(y^p_1,y^p_2) y^​2p​=max(y1p​,y2p​)

2. 計算 B g B^g Bg的面積: A g = ( x 2 g − x 1 g ) × ( y 2 g − y 1 g ) A^g = (x^g_2 - x^g_1)\times(y^g_2 - y^g_1) Ag=(x2g​−x1g​)×(y2g​−y1g​)
3. 計算 B p B^p Bp的面積: A p = ( x ^ 2 p − x ^ 1 p ) × ( y ^ 2 p − y ^ 1 p ) A^p = (\hat{x}^p_2 - \hat{x}^p_1)\times(\hat{y}^p_2 - \hat{y}^p_1) Ap=(x^2p​−x^1p​)×(y^​2p​−y^​1p​)

4. 在 B p B^p Bp 和 B g B^g Bg之間計算交集 I \mathcal{I} I :

   x 1 I = max ⁡ ( x ^ 1 p , x 1 g ) x^{\mathcal{I}}_1 = \max(\hat{x}^p_1,x^g_1) x1I​=max(x^1p​,x1g​),

   x 2 I = min ⁡ ( x ^ 2 p , x 2 g ) x^{\mathcal{I}}_2 = \min(\hat{x}^p_2,x^g_2) x2I​=min(x^2p​,x2g​),

   y 1 I = max ⁡ ( y ^ 1 p , y 1 g ) y^{\mathcal{I}}_1 = \max(\hat{y}^p_1,y^g_1) y1I​=max(y^​1p​,y1g​),

   y 2 I = min ⁡ ( y ^ 2 p , y 2 g ) y^{\mathcal{I}}_2 = \min(\hat{y}^p_2,y^g_2) y2I​=min(y^​2p​,y2g​),

   I = { ( x 2 I − x 1 I ) × ( y 2 I − y 1 I ) if x 2 I > x 1 I , y 2 I > y 1 I , 0 otherwise \mathcal{I} = \begin{cases} (x^{\mathcal{I}}_2 - x^{\mathcal{I}}_1) \times (y^{\mathcal{I}}_2 - y^{\mathcal{I}}_1) & \text{if} \quad x^{\mathcal{I}}_2 > x^{\mathcal{I}}_1, y^{\mathcal{I}}_2 > y^{\mathcal{I}}_1, \\ 0 & \text{otherwise} \end{cases} I={(x2I​−x1I​)×(y2I​−y1I​)0​ifx2I​>x1I​,y2I​>y1I​,otherwise​

5. 尋找最小封閉框（smallest enclosing box，可看下圖更清楚）的坐标 B c B^c Bc:

   x 1 c = min ⁡ ( x ^ 1 p , x 1 g ) x^{c}_1 = \min(\hat{x}^p_1,x^g_1) x1c​=min(x^1p​,x1g​),

   x 2 c = max ⁡ ( x ^ 2 p , x 2 g ) x^{c}_2 = \max(\hat{x}^p_2,x^g_2) x2c​=max(x^2p​,x2g​),

   y 1 c = min ⁡ ( y ^ 1 p , y 1 g ) y^{c}_1 = \min(\hat{y}^p_1,y^g_1) y1c​=min(y^​1p​,y1g​),

   y 2 c = max ⁡ ( y ^ 2 p , y 2 g ) y^{c}_2 = \max(\hat{y}^p_2,y^g_2) y2c​=max(y^​2p​,y2g​)

6. 計算 B c B^c Bc的面積: A c = ( x 2 c − x 1 c ) × ( y 2 c − y 1 c ) A^c = (x^c_2 - x^c_1)\times(y^c_2 - y^c_1) Ac=(x2c​−x1c​)×(y2c​−y1c​)
7. I o U = I U \displaystyle IoU = \frac{\mathcal{I}}{\mathcal{U}} IoU=UI​, where U = A p + A g − I \mathcal{U} = A^p+A^g-\mathcal{I} U=Ap+Ag−I
8. G I o U = I o U − A c − U A c \displaystyle GIoU = IoU - \frac{A^c-\mathcal{U}}{A^c} GIoU=IoU−AcAc−U​
9. L I o U = 1 − I o U \mathcal{L}_{IoU} = 1 - IoU LIoU​=1−IoU, L G I o U = 1 − G I o U \mathcal{L}_{GIoU} = 1 - GIoU LGIoU​=1−GIoU

用圖說明的更清楚

IoU=Jaccard Index

最小的封閉框是如何計算的

在 B p B^p Bp 和 B g B^g Bg之間計算黃色的交集 I \mathcal{I} I，綠色邊框表示最小的封閉框 B c B^c Bc。

最小封閉框=C

代碼實作

關于IoU Loss

根據論文UnitBox和論文GIoU對與IoU Loss處理不同的方法

UnitBox的是-ln(IoU) ln是以e為底的對數

圖上的坐标tblr的表示方式是這裡的第三種Center-Size coordinates

y i = log ⁡ e ( x i ) y_{i} = \log_{e} (x_{i}) yi​=loge​(xi​)

輸出是 x i x_{i} xi​ 輸出是 y i y_{i} yi​

GIoU裡的是1-IoU

是以代碼實作的時候，可以同時實作三個Loss

參考

UnitBox: An Advanced Object Detection Network

代碼中的坐标表示方法采用了這裡中的第一種boundary coordinates (x_min, y_min, x_max, y_max)

0,1,2,3下标可表示left，top，right，bottom

x2>x1,y2>y1

import torch
import torch.nn as nn
class IoULoss(nn.Module):
    """
    Intersetion Over Union (IoU) loss 支援三種不同的loss計算方法:
    * IoU(UnitBox paper)
    * Linear IoU(GIoU paper)
    * gIoU
    * 類型支援：iou,linear_iou,giou
    """
    def __init__(self, loc_loss_type='giou'):
        super(IoULoss, self).__init__()
        self.loc_loss_type = loc_loss_type

    def forward(self, pred, gt, weight=None):
        """
        Args:
            pred: Nx4 predicted bounding boxes, Each row is (x1, y1, x2, y2).
            gt: Nx4 gt bounding boxes
        """
        pred_x1 = pred[:, 0]
        print(pred_x1)
        pred_y1 = pred[:, 1]
        pred_x2 = pred[:, 2]
        pred_y2 = pred[:, 3]

        gt_x1 = gt[:, 0]
        gt_y1 = gt[:, 1]
        gt_x2 = gt[:, 2]
        gt_y2 = gt[:, 3]
        #如果再嚴謹些,代碼確定x2>x1,y2>y1，下标0和下标2，誰小誰是x1

        gt_aera = (gt_x1 + gt_x2) * (gt_y1 + gt_y2) #對應算法第2步
        pred_aera = (pred_x1 + pred_x2) * (pred_y1 + pred_y2)#對應算法第3步

        I_x1 = torch.max(pred_x1, gt_x1)
        I_x2 = torch.min(pred_x2, gt_x2)
        I_y1 = torch.max(pred_y1, gt_y1)
        I_y2 = torch.min(pred_x2, gt_x2)
        area_intersect=(I_x2 - I_x1)*(I_y2-I_y1)#交集 對應算法第4步

        C_x1 = torch.min(pred_x1, gt_x1)
        C_x2 = torch.max(pred_x2, gt_x2)
        C_y1 = torch.min(pred_y1, gt_y1)
        C_y2 = torch.max(pred_x2, gt_x2)
        ac =(C_x2 - C_x1) * (C_y2 - C_y1)#最小封閉框 #對應算法第5步

        U = gt_aera + pred_aera - area_intersect#并集

        ious = (area_intersect ) / (U.clamp(min=1e-10))#分母不為0
        gious = ious - (ac - U) / ac.clamp(min=1e-10)
        if self.loc_loss_type == 'iou':
            losses = -torch.log(ious)
        elif self.loc_loss_type == 'linear_iou':
            losses = 1 - ious
        elif self.loc_loss_type == 'giou':
            losses = 1 - gious
        else:
            raise NotImplementedError

        if weight is not None:
            return (losses * weight).sum()
        else:
            return losses.sum()
           

GIoU官方實作代碼