The :class:`DorefaDense` class is a binary fully connected layer, which weights are 'bitW' bits and the output of the previous layer are 'bitA' bits while inferencing. Note that, the bias vector would not be binarized. Parameters ---------- bitW : int The bits of this l
| 15 | |
| 16 | |
| 17 | class DorefaDense(Layer): |
| 18 | """The :class:`DorefaDense` class is a binary fully connected layer, which weights are 'bitW' bits and the output of the previous layer |
| 19 | are 'bitA' bits while inferencing. |
| 20 | |
| 21 | Note that, the bias vector would not be binarized. |
| 22 | |
| 23 | Parameters |
| 24 | ---------- |
| 25 | bitW : int |
| 26 | The bits of this layer's parameter |
| 27 | bitA : int |
| 28 | The bits of the output of previous layer |
| 29 | n_units : int |
| 30 | The number of units of this layer. |
| 31 | act : activation function |
| 32 | The activation function of this layer, usually set to ``tf.act.sign`` or apply :class:`Sign` after :class:`BatchNorm`. |
| 33 | use_gemm : boolean |
| 34 | If True, use gemm instead of ``tf.matmul`` for inferencing. (TODO). |
| 35 | W_init : initializer |
| 36 | The initializer for the weight matrix. |
| 37 | b_init : initializer or None |
| 38 | The initializer for the bias vector. If None, skip biases. |
| 39 | in_channels: int |
| 40 | The number of channels of the previous layer. |
| 41 | If None, it will be automatically detected when the layer is forwarded for the first time. |
| 42 | name : a str |
| 43 | A unique layer name. |
| 44 | |
| 45 | """ |
| 46 | |
| 47 | def __init__( |
| 48 | self, |
| 49 | bitW=1, |
| 50 | bitA=3, |
| 51 | n_units=100, |
| 52 | act=None, |
| 53 | use_gemm=False, |
| 54 | W_init=tl.initializers.truncated_normal(stddev=0.05), |
| 55 | b_init=tl.initializers.constant(value=0.0), |
| 56 | in_channels=None, |
| 57 | name=None, #'dorefa_dense', |
| 58 | ): |
| 59 | super().__init__(name, act=act) |
| 60 | self.bitW = bitW |
| 61 | self.bitA = bitA |
| 62 | self.n_units = n_units |
| 63 | self.use_gemm = use_gemm |
| 64 | self.W_init = W_init |
| 65 | self.b_init = b_init |
| 66 | self.in_channels = in_channels |
| 67 | |
| 68 | if self.in_channels is not None: |
| 69 | self.build((None, self.in_channels)) |
| 70 | self._built = True |
| 71 | |
| 72 | logging.info( |
| 73 | "DorefaDense %s: %d %s" % |
| 74 | (self.name, n_units, self.act.__name__ if self.act is not None else 'No Activation') |
no outgoing calls
searching dependent graphs…