Skip to content

Missing events from new databases (MySQL 8) #405

Open
@raven-kg

Description

@raven-kg

Hi there,
After upgrading my MySQL server from 5.7 to 8.0, I had a problem that events from newly created databases do not appear in the data read by the binlog reader. I used this simple code to test this and found that the data does not appear in the output if the database (table?) is created when this code is already running. And they does appear after if the code is restarted, but only the ones that occur after a restart.

import config as cfg
import logging
from pymysqlreplication import BinLogStreamReader
from pymysqlreplication.row_event import (
    DeleteRowsEvent,
    UpdateRowsEvent,
    WriteRowsEvent,
)

while True:
    try:
        stream = BinLogStreamReader(
            connection_settings={'host': cfg.client.host, 'user': cfg.client.user, 'passwd': cfg.client.password},
            server_id=11111, blocking=True, resume_stream=True, only_events=[DeleteRowsEvent, WriteRowsEvent, UpdateRowsEvent])
        for event in stream:
            if event.table != 'posts':
                continue
            for row in event.rows:
                print(row)
    except Exception as e:
        logging.exception(e)

MySQL server is AWS RDS instance, binlog/gtid settings are below:

log_bin = ON
binlog_format  = ROW
binlog_row_image  = FULL
sync_binlog  = 0
...
enforce_gtid_consistency = OFF
gtid_mode  = OFF_PERMISSIVE

mysql-replication is also upgraded to latest version

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions