Skip to content
This repository was archived by the owner on Oct 29, 2024. It is now read-only.

DataFrameClient retry on batches #378

Closed
dirkdevriendt opened this issue Oct 19, 2016 · 4 comments · Fixed by #508
Closed

DataFrameClient retry on batches #378

dirkdevriendt opened this issue Oct 19, 2016 · 4 comments · Fixed by #508

Comments

@dirkdevriendt
Copy link

When writing medium to large dataframes (multiple 50K datapoints and up) we sometimes run into server errors (HTTP 500, timeout etc). I understand this could be considered outside of the scope for the client library, but if the error is caught outside of the batch loop we can't know what data was uploaded and have to restart the upload for the entire dataframe.

Ideally, when using the the batch_size option on the DataFrameClient.write_points function it would be aware of these errors and retry a batch if needed. There are a few retry decorators out there that could be of use (e.g. https://github.com/d0ugal/retrace).

Any thoughts? Would this be a valuable addition? Would you mind taking a dependency on a retry decorator?

@geethanjalieswaran
Copy link

+1

@rterbush
Copy link

+1, just running into this issue

500 error is leaving influxdb server spinning off and taking the server down with it as a result.

@rafaelcapucho
Copy link

+1

@xginn8
Copy link
Collaborator

xginn8 commented Sep 20, 2017

I added retry logic on ConnectionError a few months back, I'll implement something similar for HTTPErrors with exponential backoff (see #435).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants