Add support for SHA256 auth plugin #583

elemount · 2017-06-23T06:46:25Z

For issue #532
SHA256 authentication plugin is the recommended auth way for the MySQL now, for security reason.
This covers all the SHA256 auth code pass.

mysqld with default_auth_plugin=sha256_password, and use SSL, server say sha256, client and server get SSL, and then client response sha256 password directly.
mysqld with default_auth_plugin=sha256_password, and do not use SSL, server say sha256, client response sha256, server (1) auth switch (2) tell the RSA public key directly, then client response the encrypted password.
mysqld with default_auth_plugin=mysql_native_password(By default), and use SSL, server say native, client and server get SSL, and then client response native, and server say AuthSwith, the client reponse the password directly.
mysqld with default_auth_plugin=mysql_native_password(By default), and do not use SSL, server say native, client say native, server say AuthSwith, and the client get RSA(already have or request '1' then server response public key) and response the encrypted password.

I test code in both 4 scenario. But can not add a proper unit test for it now. May need some help.

methane · 2017-06-23T07:00:41Z

I don't want to add dependency

elemount · 2017-06-23T07:44:53Z

@methane I use

try:
    from cryptography.hazmat.backends import default_backend
    from cryptography.hazmat.primitives import serialization, hashes
    from cryptography.hazmat.primitives.asymmetric import padding
    HAVE_CRYPTOGRAPHY = True
except ImportError:
    HAVE_CRYPTOGRAPHY = False

to make sure this module is not required.

methane · 2017-06-27T11:43:12Z

Can you use hashlib's sha256?

elemount · 2017-06-28T01:42:22Z

Sure, let me check this lib.

elemount · 2017-06-28T02:12:20Z

@methane , I'm afraid of that I can not use hashlib instead of cryptography.
The sha256_password auth method is not a hash algorithm to encrypt secret, it actually use RSA to encrypted password, in detail, it need to use RSA_PKCS1_OAEP_PADDING algorithm to encrypt the password.
In hashlib lib, its all available algorithm is

In [2]: hashlib.algorithms_available
Out[2]: 
{'DSA',
 'DSA-SHA',
 'MD4',
 'MD5',
 'RIPEMD160',
 'SHA',
 'SHA1',
 'SHA224',
 'SHA256',
 'SHA384',
 'SHA512',
 'dsaEncryption',
 'dsaWithSHA',
 'ecdsa-with-SHA1',
 'md4',
 'md5',
 'ripemd160',
 'sha',
 'sha1',
 'sha224',
 'sha256',
 'sha384',
 'sha512',
 'whirlpool'}

which do not have any algorithm which is related with RSA.
If you do not like cryptography, I can use PyCrypto instead of cryptography, or any other lib which you propose.

methane · 2017-06-29T09:40:45Z

If you do not like cryptography, I can use PyCrypto instead of cryptography, or any other lib which you propose.
View changes

I don't dislike cryptography. I just don't want any dependency.

methane · 2017-06-29T09:41:57Z

As far as I read the manual, RSA is not required when tls is used.
And binary distribution of MySQL Community (GPL) Edition doesn't support RSA.

methane · 2017-06-29T09:44:58Z

And I don't want to add any new code to connections.py. It's toooo large already.
Please move your code to pymysql/auth/sha256password.py.

elemount · 2017-06-29T10:05:20Z

@methane You are right, RSA works when the connection is not using SSL. And only built with OpenSSL but not YaSSL(which is default), this mechanism will works. I'll make sure that when using SSL, the exception will not thrown, and when the MySQL server communicate with RSA, it will throw the module needed, or you have any other suggestion?

For the connection.py code is too heavy, sure, let me move it to pymysql/auth/sha256password.py.

methane · 2017-06-29T10:18:43Z

pymysql/connections.py

+
+def _sha256_rsa_crypt(password, salt, public_key):
+    if not HAVE_CRYPTOGRAPHY:
+        raise NotImplementedError("cryptography module not found")


Please use OperationalError instead of NotImplementedError.

methane · 2017-06-29T10:20:42Z

pymysql/connections.py

+def _xor_password(password, salt):
+    password_bytes = bytearray(password, 'ascii')
+    for i in range(len(password_bytes)):
+        password_bytes[i] ^= ord(salt[i % len(salt)])


Write salt_len = len(salt) before loop and use it.

methane · 2017-06-29T10:21:33Z

pymysql/connections.py

+    password_bytes = bytearray(password, 'ascii')
+    for i in range(len(password_bytes)):
+        password_bytes[i] ^= ord(salt[i % len(salt)])
+    return password_bytes.decode('ascii')


This may cause UnicodeDecodeError. Keep bytearray type as-is.

methane · 2017-06-29T10:23:19Z

pymysql/connections.py

@@ -374,6 +396,9 @@ def is_auth_switch_request(self):
        # http://dev.mysql.com/doc/internals/en/connection-phase-packets.html#packet-Protocol::AuthSwitchRequest
        return self._data[0:1] == b'\xfe'

+    def is_rsa_public_key(self):
+        return self._data[0:1] == b'\x01'


Is \x01 really RSA_PUBLIC_KEY packet?
Isn't it a more general purpose packet which has other semantics in other context?

I think it is more general purpose packet, but I can not find a proper name, I'll fix it by the next commit for a proper packet name.

I found the real meaning of \x01, it means Extra Authentication Data.

methane · 2017-06-29T10:24:26Z

Since Oracle removes server-client protocol document, it's hard to review protocol...
Why Oracle remove such an very important document....

elemount · 2017-06-29T11:36:06Z

I use this web archived document https://web.archive.org/web/20170404163410/https://dev.mysql.com/doc/internals/en/sha256.html .
This is another link which is helpful https://dev.mysql.com/doc/refman/5.7/en/sha256-pluggable-authentication.html .

methane · 2017-06-29T12:22:37Z

I know webarchive, but all images (including important sequence diagram) gone.

elemount · 2017-07-04T05:40:54Z

@methane Could you help to review it again?

elemount · 2017-08-10T02:44:03Z

Hi @methane ,I squashed this PR and fixed up the tests. Could you help to review it?

sscherfke · 2017-11-09T10:29:03Z

You can use extras_require in setup.py to declare cryptography as optional dependency, e.g.,

setup(
    name="PyMySQL",
    # ...
    extras_require={
        'auth_sha256': ['cryptography>=2.1'],
    },
    classifiers=[
        # ...
    ],
)

Users can then pip install pymysql w/o dependenies and pip install pymysql[auth_sha256] with dependencies.

methane · 2018-03-19T11:11:32Z

@elemount Why did you close this? May I continue your work?

elemount · 2018-04-12T08:00:44Z

I closed it because I abandoned using SHA256 in my MySQL server.
You can continue my work with current code. I believe these code can work.

grooverdan · 2018-04-29T07:43:35Z

@elemount, thanks for your work. I've started to look at this. Overall looks pretty good and I've only added a couple of Py3 compatibility fixes so far.
server_public_key looks like raw bytes read from the server but a filename serialization.load_pem_public_key when passed from to the connection. I haven't thought yet how to resolve this.

Hope the auth plugin interface wasn't too bad/buggy.

WIP:

grooverdan/PyMySQL@travis-docker-mysql-8.0...grooverdan:sha256_auth_plugin

Next steps:
reenable existing plugin tests (on tcp, unix_socket tests appear disabled)
write more tests (thankfully MySQL-8.0 seems to generate and configure client/server ssl keys as well).
implement the caching mechanism and check all the paths are implemented: https://dev.mysql.com/doc/dev/mysql-server/latest/page_caching_sha2_authentication_exchanges.html

methane · 2018-04-30T06:22:27Z

FYI, I'll release a few bugfix (0.8.x) in May.
Large changes including this should be merged after that. (Maybe, June)

elemount mentioned this pull request Jun 23, 2017

SHA256 user account issue #532

Closed

bgrainger mentioned this pull request Jun 27, 2017

Add support for sha256_password mysql-net/MySqlConnector#281

Closed

methane reviewed Jun 29, 2017

View reviewed changes

elemount added 2 commits August 10, 2017 02:23

Add support for SHA256 auth plugin

695df63

travis: MySQL 5.6.35 => 5.6.37, MySQL 5.7.17 => 5.7.19

c93b322

elemount closed this Nov 21, 2017

elemount reopened this Apr 12, 2018

grooverdan mentioned this pull request Apr 22, 2018

The password is missing #650

Closed

twiddli mentioned this pull request May 21, 2018

Hello When I connect the MySQL8.0 times error with pymysql. #674

Closed

Merge branch 'master' into master

803bdca

methane mentioned this pull request Jun 8, 2018

Add sha256 and chaching sha2 auth support #682

Merged

methane closed this Jun 8, 2018

zzzeek mentioned this pull request Mar 14, 2019

add ed22519 support #786

Closed

4 tasks

github-actions bot locked as resolved and limited conversation to collaborators Aug 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for SHA256 auth plugin #583

Add support for SHA256 auth plugin #583

elemount commented Jun 23, 2017 •

edited

Loading

methane commented Jun 23, 2017

elemount commented Jun 23, 2017

methane commented Jun 27, 2017

elemount commented Jun 28, 2017

elemount commented Jun 28, 2017 •

edited

Loading

methane commented Jun 29, 2017

methane commented Jun 29, 2017

methane commented Jun 29, 2017

elemount commented Jun 29, 2017

methane Jun 29, 2017

elemount Jun 29, 2017

methane Jun 29, 2017

elemount Jun 29, 2017

methane Jun 29, 2017

elemount Jun 29, 2017

methane Jun 29, 2017

elemount Jun 29, 2017

elemount Jun 30, 2017

methane commented Jun 29, 2017

elemount commented Jun 29, 2017

methane commented Jun 29, 2017

elemount commented Jul 4, 2017

elemount commented Aug 10, 2017

sscherfke commented Nov 9, 2017 •

edited

Loading

methane commented Mar 19, 2018

elemount commented Apr 12, 2018 •

edited

Loading

grooverdan commented Apr 29, 2018

methane commented Apr 30, 2018

Add support for SHA256 auth plugin #583

Add support for SHA256 auth plugin #583

Conversation

elemount commented Jun 23, 2017 • edited Loading

methane commented Jun 23, 2017

elemount commented Jun 23, 2017

methane commented Jun 27, 2017

elemount commented Jun 28, 2017

elemount commented Jun 28, 2017 • edited Loading

methane commented Jun 29, 2017

methane commented Jun 29, 2017

methane commented Jun 29, 2017

elemount commented Jun 29, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

methane commented Jun 29, 2017

elemount commented Jun 29, 2017

methane commented Jun 29, 2017

elemount commented Jul 4, 2017

elemount commented Aug 10, 2017

sscherfke commented Nov 9, 2017 • edited Loading

methane commented Mar 19, 2018

elemount commented Apr 12, 2018 • edited Loading

grooverdan commented Apr 29, 2018

methane commented Apr 30, 2018

elemount commented Jun 23, 2017 •

edited

Loading

elemount commented Jun 28, 2017 •

edited

Loading

sscherfke commented Nov 9, 2017 •

edited

Loading

elemount commented Apr 12, 2018 •

edited

Loading