-
-
Notifications
You must be signed in to change notification settings - Fork 7.9k
Fix #19128: webagg reports incorrect values for non-alphanumeric key events on non-qwerty keyboards #19146
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix #19128: webagg reports incorrect values for non-alphanumeric key events on non-qwerty keyboards #19146
Conversation
if (event.shiftKey && event.which !== 16) { | ||
value += 'shift+'; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is shift never added now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not any more.
_handle_key(key)
was the one that required the shift+
string on the key value. If shift+
was on a key value, then the function removed the shift+
of the string and depending on the remaining of the key code, it got the "correct" value (it was correct only if the user had a QWERTY keyboard and a US keyboard layout). e.g. "shift+2" was replaced with "@".
As KeyboardEvent.key
handles that for us (even better, with support for non US keyboard layouts), we no longer need to specify the shift+
modifier on a key value.
Does this correctly handle |
If you press python -c 'from pylab import *; plot(); gcf().canvas.mpl_connect("key_press_event", lambda e: print(e.key)); show()' It prints:
but not |
I get different behavior with qt and (on the default branch) webagg
|
I get the same behaviour on Gtk3Agg and TkAgg...
|
Ah, so we have identified an inconsistency between Qt and {tk, gtk}, that is exciting (in the wrong sort of ways)! Both tk and qt seem to tack ctrl on the front of the f-keys. jupiter@14:05 ➤ MPLBACKEND=Qt5Agg python -c 'from pylab import *; plot(); gcf().canvas.mpl_connect("key_press_event", lambda e: print(e.key)); show()'
control
ctrl+f1
ctrl+f1
q
jupiter@14:09 ➤ MPLBACKEND=TkAgg python -c 'from pylab import *; plot(); gcf().canvas.mpl_connect("key_press_event", lambda e: print(e.key)); show()'
control
ctrl+f1
ctrl+f1
ctrl+f1
control
ctrl+shift
shift
control
ctrl+f1
ctrl+f1
ctrl+f1 so I guess the question is if we should treat the shift key as a "modfier" key that we inform the user about when the key being shifted does not have a "natural" shift value (I guess we could return F1 vs f1? (that is a joke)). I think the way Qt is doing it (to treat shift as a sometimes mofidier key) is more correct, but am not sure of that. Given that there is inconsistency between the backends already, my preference would be for webagg to "do what it did before", but would not block merging over that behavior being changed (as long as it is documented). @tonadev Bonus points if you make all of the backends consistent in this PR, but if not can you please write up an issue with your thoughts / findings on the differences? |
Also, welcome to the project and congratulations on finding a can of worms on your first try ;) |
I'll try to make all of the backends consistent in this PR. I have already started working on that. This is what I have found so far. The following are the result of pressing some "random" key combinations on different backends using a non-US keyboard layout (that's why the correct value for "Shift+2" is ("), there is no need to test all the different Qt, Gtk, etc. backends as they share the same behaviour on keyboard events) on the master branch.
|
What do you mean with "a sometimes modifier key"? Maybe, we should add "shift+" to a key value, only if Wx backend key event handling is very poor, as noted on the table on my previous comment, it doesn't handle shift plus letter/number/symbol keys correctly. Must be improved.
Thanks! |
As of my last changes, these are the results of pressing the same key combinations of one of my previous comments.
There is still inconsistency on some key combinations. We can observe that Wx and Qt fail on handling uppercase letters. This is due to the fact that these libraries (Wx and Qt) don't treat Caps Lock as a modifier (for us to know if it's on, we would require to use OS specific functions) and every time the user enters a letter it gives us the letter in uppercase, making it difficult to us to know if we should handle it as an uppercase letter or a lowercase letter. We can treat letters as if they were uppercase always too, to get even more consistency. We can also observe that Wx keeps failing on handling |
@tonadev That is an amazing table! To make sure i am understanding it right, when you write "+ Enter +" you mean there is another line? I think what is going on here is that for e.g. "Ctrl+f1" we are processing two key events, first that ctrl has been pressed and then a second that f1 was pressed while control was held down. What I meant "What do you mean with "a sometimes modifier key"?" is that for keys that have "an upper case" ('a' vs 'A') we should return the upper case letter ("A") not the modifier and lower case ("shift + a"). |
In things that are fun, it seems that the order in which you hit the modifiers matters (?!) It seems that "shifted alt" is "super" or "meta" (I am testing on a branch without these patches with Qt) (shift then alt)
(alt then shift)
I also do not get an event for super being pushed with Qt I do not think I have an AltGr on my keboard layout. If you use it in combination with other keys do you get the glyphs out that you expect? I think part of the tension here is that the keyboard input abstraction is leaky with various parts of it being handled at the physical, OS, window manager, GUI toolkit, and application layers. |
@tonadev You may also be interested in joining https://gitter.im/matplotlib/incubator |
Yes!, that's right.
I think I got it. |
That's true. Actually, that behaviour is consistent between our backends. As a matter of fact, the docstring of
It seems that the other backends do the same. You can see that on the row for
Right now, we are not supporting the super key on Qt.
I do (Alt Gr = Alt Graph = the alt to the right of the spacebar). |
I'll use this comment to keep track of the behaviour of the different backends up to date.
|
@QuLogic @tacaswell Well... I think I got stuck. Most of the remaining inconsistencies are due to limitations by the backend library. For example,
If we handle On Qt and Wx using And on Tk, when pressing Any suggestions? |
If there are limitations/inconsistencies in the GUI toolkits then there are inconsistencies. You did not introduce them and the have probably been there from the beginning so we should take the win of the stuff you have fixed and document what is not working (that table should probably go in to https://matplotlib.org/users/event_handling.html / https://github.com/matplotlib/matplotlib/blob/master/doc/users/event_handling.rst). |
The *matplotlib.backends.backend_qt5.SPECIAL_KEYS* dictionary | ||
contains keys that do *not* return their unicode name | ||
instead they have manually specified names. The name | ||
for *QtCore.Qt.Key_Meta* has changed to ``'meta'`` to be consistent. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for *QtCore.Qt.Key_Meta* has changed to ``'meta'`` to be consistent. | |
for *QtCore.Qt.Key_Meta* has changed to ``'meta'`` to be consistent with the | |
other GUI backends. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this instead be titled "harmonized key event data across backends" or something like that? That way it can also note the changes to shift
labeling and the fixes to webagg.
I agree these changes are definitely a bug fix, I would bet that there exists more than one user with a non-US keyboard who discovered what keys actually got sent when they hit "shift+2" and code against that (I am confident about this because I also use a non-qwerty layout and have some things configured in these double-negative ways) . This will break them so we should note it, particularly because this is something where there is no (reasonable) way to do warnings or provide an easy way to get back to the old behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't a lot of experience writing documentation nor api changes, also, English is not my main language.
@tacaswell please can you take a look at my lastest commits and don't hesitate to do observations about typos/grammar, formatting, etc.
[event.AltDown, 'alt', 'alt'], | ||
[event.ShiftDown, 'shift', 'shift'],): | ||
if meth() and not key_name == key: | ||
if not (key_name == 'shift' and key.isupper()): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks like it just gives up on non-ascii (?!) base on the if...elif...else
just above this? Am I reading that right?
I think fixing that is out of scope for this PR (to keep it from getting to big, small PRs are easier to review and get in).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
key = chr(keyval)
# wx always returns an uppercase, so make it lowercase if the shift
# key is not depressed (NOTE: this will not handle Caps Lock)
if not event.ShiftDown():
key = key.lower()
Thanks to this fragment of code, we get a lowercase letter if shift is not pressed. If it's, then we get an uppercase letter.
Then, the following line of code makes sure we don't get shift+A
and rather get shift
, enter and A
.
if not (key_name == 'shift' and key.isupper()):
How ever we are still interested on getting shift+2
or shift+]
as Wx doesn't translate the keys. Hence only when shift was pressed and the key is an uppercase letter we are interested in not adding the shift prefix. If we don't do this, then there would be one more inconsistency on Wx backend.
This is great work @tonadev ! |
============== ============================= ============================== ============================= ============================== ============================== | ||
Key(s) Pressed WxPython Qt WebAgg Gtk Tkinter | ||
============== ============================= ============================== ============================= ============================== ============================== | ||
Shift+2 shift, shift+2 shift, " shift, " shift, " shift, " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What layout are you using? shift+2 is @ on US QWERTY keyboards.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A Spanish QWERTY keyboard. Sorry! I forgot to consider that fact.
A a a A A A | ||
a a a a a a | ||
Shift+a shift, A shift, A shift, A shift, A shift, A | ||
Shift+A shift, A shift, A shift, a shift, a shift, a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What keyboard has a capital A already? Do you mean with caps lock on?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I mean with caps lock on.
|
||
if not (prefix == 'shift' and unikey): | ||
key = '{0}+{1}'.format(prefix, key) | ||
break |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doesn't this break mean you can only add one modifier?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we hit ctrl + shift + alt, then we get ctrl, ctrl + shift and finally ctrl + alt. I noticed it, but haven't been able to work on this PR since I've been a little busy. If someone else want to continue with the work, go ahead! If not, I will work on this when I get more time. I'm sorry for the inconveniences.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Assuming you meant ctrl+shift+alt for the last one, I think that is correct behavior, isn't it?
…es on keyboard events between user interface tookits
Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>
Co-authored-by: Elliott Sales de Andrade <quantum.analyst@gmail.com>
@tonadev I took the liberty of rebasing, consolidating the API changes notes, and force-pushing to your branch hope you do not mind! |
Thanks @tonadev! Congratulations on your first PR to Matplotlib 🎉 We hope to hear from you again. |
ports changes from matplotlib/matplotlib#19146 closes matplotlib#309
ports changes from matplotlib/matplotlib#19146 closes matplotlib#309
ports changes from matplotlib/matplotlib#19146 closes matplotlib#309
PR Summary
Fixes #19128. This PR improves webagg support for non-alphanumeric key events on non-qwerty keyboards.
KeyboardEvent.which
got deprecated and Mozilla recommends to useKeyboardEvent.key
instead.KeyboardEvent.key
works different:this allows us to handle correctly combinations of modifiers and keys, even on non-qwerty keyboards.
As I couldn't find a full list of all Matplotlib key names (however the class
KeyEvent
on backend_bases.py was a bit useful), I tried to keep support for all the keys supported before this changes (those on_LUT
and_SHIFT_LUT
) by using the same key names.I also use the same code for reproduction of the bug reported on #19128 on the GTK3Agg backend as example of how key events should have been handled.
PR Checklist
pytest
passes).flake8
on changed files to check).flake8-docstrings
and runflake8 --docstring-convention=all
).doc/users/next_whats_new/
(follow instructions in README.rst there).doc/api/next_api_changes/
(follow instructions in README.rst there).