typical for M-RoPE cases: 0 - sequantial position of the tokens/embeddings in the sequence 1 - y position in the image 2 - x position in the image 3 - other
| 23 | // 2 - x position in the image |
| 24 | // 3 - other |
| 25 | bool is_pos_2d() const { |
| 26 | // TODO @ngxson : we may need to check for model arch when more models use >1 positions |
| 27 | return n_pos >= 3; |
| 28 | } |
| 29 | |
| 30 | uint32_t b_equal_seqs; // note: this is a boolean, but we use an int32_t for alignment |
| 31 | // otherwise address sanitizer complains |
no outgoing calls
no test coverage detected