Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... S
(x, drop_prob=0., training=False, scale_by_keep=True)
| 405 | |
| 406 | |
| 407 | def drop_path(x, drop_prob=0., training=False, scale_by_keep=True): |
| 408 | """ Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). |
| 409 | |
| 410 | This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, |
| 411 | the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... |
| 412 | See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for |
| 413 | changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use |
| 414 | 'survival rate' as the argument. |
| 415 | |
| 416 | """ |
| 417 | if drop_prob == 0. or not training: |
| 418 | return x |
| 419 | keep_prob = 1 - drop_prob |
| 420 | shape = (x.shape[0],) + (1,) * (x.ndim - 1) # work with diff dim tensors, not just 2D ConvNets |
| 421 | random_tensor = x.new_empty(shape).bernoulli_(keep_prob) |
| 422 | if keep_prob > 0.0 and scale_by_keep: |
| 423 | random_tensor.div_(keep_prob) |
| 424 | return x * random_tensor |
| 425 | |
| 426 | |
| 427 | class DropPath(nn.Module): |