Method
__init__
(self, hidden_size, cross_attention_dim=None, num_tokens=4, scale=1.0)
Source from the content-addressed store, hash-verified
| 2003 | """ |
| 2004 | |
| 2005 | def __init__(self, hidden_size, cross_attention_dim=None, num_tokens=4, scale=1.0): |
| 2006 | super().__init__() |
| 2007 | |
| 2008 | self.hidden_size = hidden_size |
| 2009 | self.cross_attention_dim = cross_attention_dim |
| 2010 | self.num_tokens = num_tokens |
| 2011 | self.scale = scale |
| 2012 | |
| 2013 | self.to_k_ip = nn.Linear(cross_attention_dim or hidden_size, hidden_size, bias=False) |
| 2014 | self.to_v_ip = nn.Linear(cross_attention_dim or hidden_size, hidden_size, bias=False) |
| 2015 | |
| 2016 | def __call__( |
| 2017 | self, |
Callers
nothing calls this directly
Tested by
no test coverage detected