Skip to content

Webhook Broken Pipe after connection reset, kills dispatcher #970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
yusufk opened this issue Jan 10, 2018 · 4 comments · Fixed by #985
Closed

Webhook Broken Pipe after connection reset, kills dispatcher #970

yusufk opened this issue Jan 10, 2018 · 4 comments · Fixed by #985
Assignees
@tsnoam
Labels
📋 pending-reply work status: pending-reply

Comments

@yusufk
Copy link

yusufk commented Jan 10, 2018 •
edited by tsnoam
Loading

Steps to reproduce

Standard Webhook with updater and dispatcher:

updater = Updater(TOKEN) PORT = int(os.environ.get('PORT', '5000')) # add handlers 
updater.start_webhook(listen="0.0.0.0", port=PORT, url_path=TOKEN) 
updater.bot.set_webhook("https://home.co.za/" + TOKEN) 
dp = updater.dispatcher dp.add_error_handler(error) 
updater.idle()

Expected behaviour

Bot should run and remain responsive

Actual behaviour

It runs, but after a few hours becomes unresponsive.

Configuration

Operating System:
Linux Cappucino 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:50 UTC 2017 i686 athlon i686 GNU/Linux

Version of Python, python-telegram-bot & dependencies:

python-telegram-bot 9.0.0
certifi 2017.11.05
future 0.16.0
Python 3.5.2 (default, Nov 23 2017, 16:37:01)  [GCC 5.4.0 20160609]

Logs

2018-01-10 03:54:56,480 - telegram.ext.updater - ERROR - unhandled exception
Traceback (most recent call last):
  File "/usr/lib/python3.5/socketserver.py", line 313, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 341, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/lib/python3.5/dist-packages/telegram/utils/webhookhandler.py", line 62, in __init__
    super(WebhookHandler, self).__init__(request, client_address, server)
  File "/usr/lib/python3.5/socketserver.py", line 681, in __init__
    self.handle()
  File "/usr/lib/python3.5/http/server.py", line 422, in handle
    self.handle_one_request()
  File "/usr/lib/python3.5/http/server.py", line 390, in handle_one_request
    self.raw_requestline = self.rfile.readline(65537)
  File "/usr/lib/python3.5/socket.py", line 575, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/telegram/ext/updater.py", line 149, in _thread_wrapper
    target(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/telegram/ext/updater.py", line 353, in _start_webhook
    self.httpd.serve_forever(poll_interval=1)
  File "/usr/local/lib/python3.5/dist-packages/telegram/utils/webhookhandler.py", line 41, in serve_forever
    super(WebhookServer, self).serve_forever(poll_interval)
  File "/usr/lib/python3.5/socketserver.py", line 234, in serve_forever
    self._handle_request_noblock()
  File "/usr/lib/python3.5/socketserver.py", line 315, in _handle_request_noblock
    self.handle_error(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 371, in handle_error
    print('Exception happened during processing of request from', end=' ')
BrokenPipeError: [Errno 32] Broken pipe
Exception in thread updater:
Traceback (most recent call last):
  File "/usr/lib/python3.5/socketserver.py", line 313, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 341, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 354, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/local/lib/python3.5/dist-packages/telegram/utils/webhookhandler.py", line 62, in __init__
    super(WebhookHandler, self).__init__(request, client_address, server)
  File "/usr/lib/python3.5/socketserver.py", line 681, in __init__
    self.handle()
  File "/usr/lib/python3.5/http/server.py", line 422, in handle
    self.handle_one_request()
  File "/usr/lib/python3.5/http/server.py", line 390, in handle_one_request
    self.raw_requestline = self.rfile.readline(65537)
  File "/usr/lib/python3.5/socket.py", line 575, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/local/lib/python3.5/dist-packages/telegram/ext/updater.py", line 149, in _thread_wrapper
    target(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/telegram/ext/updater.py", line 353, in _start_webhook
    self.httpd.serve_forever(poll_interval=1)
  File "/usr/local/lib/python3.5/dist-packages/telegram/utils/webhookhandler.py", line 41, in serve_forever
    super(WebhookServer, self).serve_forever(poll_interval)
  File "/usr/lib/python3.5/socketserver.py", line 234, in serve_forever
    self._handle_request_noblock()
  File "/usr/lib/python3.5/socketserver.py", line 315, in _handle_request_noblock
    self.handle_error(request, client_address)
  File "/usr/lib/python3.5/socketserver.py", line 371, in handle_error
    print('Exception happened during processing of request from', end=' ')
BrokenPipeError: [Errno 32] Broken pipe

2018-01-10 03:54:56,601 - telegram.ext.dispatcher - CRITICAL - stopping due to exception in another thread
2018-01-10 03:54:56,601 - telegram.ext.dispatcher - DEBUG - Dispatcher thread stopped
2018-01-10 03:54:56,601 - telegram.ext.updater - DEBUG - dispatcher - ended
@tsnoam tsnoam self-assigned this Jan 17, 2018
@tsnoam tsnoam added the bug label Jan 17, 2018
tsnoam added a commit that referenced this issue Jan 20, 2018
@tsnoam
webhookhandler: Fix exception thrown during error handling
010024f 
BaseServer.handle_error() default behaviour is to print to stdout or
stderr (depends on the python version). In case that the file descriptor
is closed an additional exception will be raised, resulting with the
webhook thread to quit.

Fixes #970
@tsnoam
Copy link
Member

tsnoam commented Jan 20, 2018

@yusufk
Can you please try to patch your bot with 010024f and see if it works for you?
That is before we merge the patch to master.

@tsnoam tsnoam added the 📋 pending-reply work status: pending-reply label Jan 20, 2018
@yusufk
Copy link
Author

yusufk commented Jan 22, 2018

Patched and running, will let it run and see what happens.

p.s. I used updater.idle(stop_signals=(2, 15, 6, 13)) to pick up broken pipe signals as a temp fix.

@tsnoam
Copy link
Member

tsnoam commented Jan 22, 2018

@yusufk Thanks. Just make sure you've disabled the work around ;-)

@yusufk
Copy link
Author

yusufk commented Jan 23, 2018

Howdy,

Seems to have done the trick! Continued functioning despite multiple connection resets etc.

Thanks
Yusuf

@tsnoam tsnoam mentioned this issue Jan 24, 2018
Merged
tsnoam added a commit that referenced this issue Jan 25, 2018
@tsnoam
webhookhandler: Fix exception thrown during error handling (#985)
8690ba2 
BaseServer.handle_error() default behaviour is to print to stdout or
stderr (depends on the python version). In case that the file descriptor
is closed an additional exception will be raised, causing the webhook
thread to quit.

Fixes #970
@github-actions github-actions bot locked and limited conversation to collaborators Aug 24, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@tsnoam tsnoam

Labels
📋 pending-reply work status: pending-reply
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

webhookhandler: Fix exception thrown during error handling python-telegram-bot/python-telegram-bot
3 participants
@yusufk @tsnoam @Bibo-Joshi