Add JSON field type and include it in TEXT_TYPES list to prevent encoding problems #477
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: This issue is not related to the following pull request: https://github.com/PyMySQL/PyMySQL/pull/428/commits
MySQL 5.7 added JSON types. In
_get_descriptions
method we are setting the encoding of any field type that is not textual to 'ascii'. This is clearly wrong as the JSON field types are actually BINARY column that contains strings.Because of wrong encoding problem, in
_read_row_from_packet
function we encode the JSON field type to 'ascii'. Here is the related code:Because of this problem, querying JSON field types that contain Unicode characters will fail.
The solution to the problem is relatively easy. Add the JSON field types to TEXT_TYPES.