auto">Came here from a conversation in #1826, looks like that the issue is bigger than I was thinking about :(

So the main issue is the state of the bot in the tests. Example (the same bot is chosen): I run test_set_chat_photo test locally, someone else run test_delete_chat_photo test locally and if I run test_delete_chat_photo locally - it will fail.

The doable option that I can see so far - use different bots for the CI tests but let developers still use the existing ones. It will reduce the flakiness in the tests by not damaging the state of the bots from local tests. And reordering of some tests can help.

Example of the tests for the master branch (28ded67 commit):

    @flaky(3, 1)
    @pytest.mark.timeout(10)
    def test_delete_chat_photo(self, bot, channel_id):
        assert bot.delete_chat_photo(channel_id)

    @flaky(3, 1)
    @pytest.mark.timeout(10)
    def test_set_chat_photo(self, bot, channel_id):
        with open('tests/data/telegram_test_channel.jpg', 'rb') as f:
            assert bot.set_chat_photo(channel_id, f)

py.test executes tests in the order they are given in the file unless something else is specified.
In this case - it is an issue. The first test tries to delete a chat photo and the second one actually set its up.

[BUG] make test bots un/pin messages in different chats to make tests more stable #1759

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions