llama : fix KV shift for qwen2vl #13870
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix #13865
Provided a model with 3 mrope sections, a rotated vector looks like this:
[time, time+x, time+y]
This works with the assumption that mrope with
x == y == 0
is identical to doing a neox rope with theta =+time
For example, when we want to shift the
time
totime-1
, that mean we shift from[time, time+x, time+y]
to[time-1, time-1+x, time-1+y]
; We can simply apply neox rope with theta =-1
to archive the same effect