The :class:`QuanDense` class is a quantized fully connected layer with BN, which weights are 'bitW' bits and the output of the previous layer are 'bitA' bits while inferencing. Parameters ---------- n_units : int The number of units of this layer. act : activation functi
| 15 | |
| 16 | |
| 17 | class QuanDense(Layer): |
| 18 | """The :class:`QuanDense` class is a quantized fully connected layer with BN, which weights are 'bitW' bits and the output of the previous layer |
| 19 | are 'bitA' bits while inferencing. |
| 20 | |
| 21 | Parameters |
| 22 | ---------- |
| 23 | n_units : int |
| 24 | The number of units of this layer. |
| 25 | act : activation function |
| 26 | The activation function of this layer. |
| 27 | bitW : int |
| 28 | The bits of this layer's parameter |
| 29 | bitA : int |
| 30 | The bits of the output of previous layer |
| 31 | use_gemm : boolean |
| 32 | If True, use gemm instead of ``tf.matmul`` for inference. (TODO). |
| 33 | W_init : initializer |
| 34 | The initializer for the weight matrix. |
| 35 | b_init : initializer or None |
| 36 | The initializer for the bias vector. If None, skip biases. |
| 37 | in_channels: int |
| 38 | The number of channels of the previous layer. |
| 39 | If None, it will be automatically detected when the layer is forwarded for the first time. |
| 40 | name : None or str |
| 41 | A unique layer name. |
| 42 | |
| 43 | """ |
| 44 | |
| 45 | def __init__( |
| 46 | self, |
| 47 | n_units=100, |
| 48 | act=None, |
| 49 | bitW=8, |
| 50 | bitA=8, |
| 51 | use_gemm=False, |
| 52 | W_init=tl.initializers.truncated_normal(stddev=0.05), |
| 53 | b_init=tl.initializers.constant(value=0.0), |
| 54 | in_channels=None, |
| 55 | name=None, #'quan_dense', |
| 56 | ): |
| 57 | super().__init__(name, act=act) |
| 58 | self.n_units = n_units |
| 59 | self.bitW = bitW |
| 60 | self.bitA = bitA |
| 61 | self.use_gemm = use_gemm |
| 62 | self.W_init = W_init |
| 63 | self.b_init = b_init |
| 64 | self.in_channels = in_channels |
| 65 | |
| 66 | if self.in_channels is not None: |
| 67 | self.build((None, self.in_channels)) |
| 68 | self._built = True |
| 69 | |
| 70 | logging.info( |
| 71 | "QuanDense %s: %d %s" % |
| 72 | (self.name, n_units, self.act.__name__ if self.act is not None else 'No Activation') |
| 73 | ) |
| 74 |
no outgoing calls
searching dependent graphs…