Init a General Matrix multiplication(Gemm) operator. Compute `Y = alpha * A' * B' + beta * C`, where input tensor A has shape (M, K) or (K, M), input tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to shape (M, N), and output tensor Y has shape (M, N). `A' =
| 3684 | |
| 3685 | |
| 3686 | class Gemm(Operator): |
| 3687 | """ |
| 3688 | Init a General Matrix multiplication(Gemm) operator. Compute `Y = alpha * |
| 3689 | A' * B' + beta * C`, where input tensor A has shape (M, K) or (K, M), input |
| 3690 | tensor B has shape (K, N) or (N, K), input tensor C is broadcastable to |
| 3691 | shape (M, N), and output tensor Y has shape (M, N). |
| 3692 | `A' = transpose(A)` if transA else A |
| 3693 | `B' = transpose(B)` if transB else B |
| 3694 | """ |
| 3695 | |
| 3696 | def __init__(self, alpha=1.0, beta=1.0, transA=0, transB=0): |
| 3697 | """ |
| 3698 | Args: |
| 3699 | alpha (float): Scalar multiplier for the product of input tensors |
| 3700 | A * B. |
| 3701 | beta (float): Scalar multiplier for input tensor C. |
| 3702 | ransA (int): Whether A should be transposed |
| 3703 | transB (int): Whether B should be transposed |
| 3704 | Returns: |
| 3705 | CTensor, the output |
| 3706 | """ |
| 3707 | super(Gemm, self).__init__() |
| 3708 | self.alpha = alpha |
| 3709 | self.beta = beta |
| 3710 | self.transA = transA |
| 3711 | self.transB = transB |
| 3712 | |
| 3713 | def forward(self, A, B, C=None): |
| 3714 | """ |
| 3715 | forward propogation of Gemm |
| 3716 | Args: |
| 3717 | A (CTensor): The shape of A should be (M, K) if transA is 0, or |
| 3718 | (K, M) if transA is non-zero. |
| 3719 | B (CTensor): The shape of B should be (K, N) if transB is 0, or |
| 3720 | (N, K) if transB is non-zero. |
| 3721 | C (CTensor): (optional), Optional input tensor C. If not specified, |
| 3722 | the computation is done as if C is a scalar 0. The shape of C |
| 3723 | should be unidirectional broadcastable to (M, N). |
| 3724 | Returns: |
| 3725 | tensor, the output |
| 3726 | """ |
| 3727 | _A = singa.DefaultTranspose(A) if self.transA == 1 else A |
| 3728 | _B = singa.DefaultTranspose(B) if self.transB == 1 else B |
| 3729 | if training: |
| 3730 | self.inputs = (_A, _B, C) |
| 3731 | tmpM = singa.MultFloat(singa.Mult(_A, _B), self.alpha) |
| 3732 | if C: |
| 3733 | tmpM = singa.__add__(tmpM, singa.MultFloat(C, self.beta)) |
| 3734 | return tmpM |
| 3735 | |
| 3736 | def backward(self, dy): |
| 3737 | """ |
| 3738 | backward propogation of Gemm |
| 3739 | Args: |
| 3740 | dy (CTensor): The shape of A should be (M, K) if transA is 0, or (K, M) if transA is non-zero. |
| 3741 | Returns: |
| 3742 | CTensor, the gradient over A |
| 3743 | CTensor, the gradient over B |