Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... Se
(
x, drop_prob: float = 0.0, training: bool = False, scale_by_keep: bool = True
)
| 136 | |
| 137 | |
| 138 | def drop_path( |
| 139 | x, drop_prob: float = 0.0, training: bool = False, scale_by_keep: bool = True |
| 140 | ): |
| 141 | """Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). |
| 142 | |
| 143 | This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, |
| 144 | the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... |
| 145 | See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for |
| 146 | changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use |
| 147 | 'survival rate' as the argument. |
| 148 | |
| 149 | """ |
| 150 | if drop_prob == 0.0 or not training: |
| 151 | return x |
| 152 | keep_prob = 1 - drop_prob |
| 153 | shape = (x.shape[0],) + (1,) * ( |
| 154 | x.ndim - 1 |
| 155 | ) # work with diff dim tensors, not just 2D ConvNets |
| 156 | random_tensor = x.new_empty(shape).bernoulli_(keep_prob) |
| 157 | if keep_prob > 0.0 and scale_by_keep: |
| 158 | random_tensor.div_(keep_prob) |
| 159 | return x * random_tensor |
| 160 | |
| 161 | |
| 162 | class DropPath(nn.Module): |