-
Notifications
You must be signed in to change notification settings - Fork 524
gzip support on influxdb-python #718
Comments
Related: #451 |
Hello, I just wanted to jump in and let you know that I've been able to add gzip compression to the client and have made a PR #732 I agree that data compression is really important, especially on IoT devices and mobile networks. I'm unsure how to properly do the tests so it may still be awhile until it's merged. |
Thank you for the response KEClaytor & driftregion, much appreciated. Will be happy to do some testing when you are ready / there is a consensus about how this should be done. |
Related: #733. |
@driftregion Yes, compression seems to be supported. Thanks for linking to the official docs on this. The earliest mention of it in the changelog is in v1.0: @lovasoa Message Pack looks really neat. The only reference I can find to it in the docs is:
From: https://docs.influxdata.com/influxdb/v1.7/about_the_project/releasenotes-changelog/ Do you know if they also support message pack for transmitting data? If it's just for the query responses then gzip'd data for transmission still makes sense. I'd prefer to see some mention of it on their official docs page, but we could probably implement it the same way I did gzip - just compress / convert the message before write. Since gzip seems to be supported back to v1.0 and for transmission and receiving that would be my preferred way to go. |
It's really great that you are all jumping in on this request, thank you. As I (inappropriately) posted on the related MessagePack issue, I think MessagePack is a great alternative to JSON, but I also suspect that it is orthogonal to the issue of using gzip for HTTPS data transmission? I suspect both JSON and MessagePack containing perhaps hundreds of data points, probably with highly repetitive field names, would be served very well by gzip compression? On that basis I would avoid conflating the issues. Happy to be corrected regarding this if I've got the wrong end of the stick. cheers, Phil. |
@KEClaytor No, msgpack doesn't seem to be supported for sending data. See the corresponding pull request. |
Hi all, I was wondering if there was any chance of moving this request forwards? (Specifically, implementing / rolling out gzip compression). IoT clients still crying out for some compression :-) kind regards & thank you, Phil. |
Oh I think this is done (I'm pretty sure this issue can be closed), I checked this issue because I was getting a weird key error related to the constructor when providing the The InfluxDBClient documentation lists client = InfluxDBClient(host, port, u, p, 'example', gzip=True) # By default in the doc, gzip=False |
@malcolmwhat After you enabled gzip on the constructor how exactly did you manage to write a gzipped file with the library? |
Hi,
Great API, we use it heavily. Thank you.
The equivalent Java API influxdb-java, which I also use, allows gzip compression to be explicitly enabled when setting up a connection, e.g.
Perhaps this is enabled by default in influxdb-python, however I cannot find any reference to gzip compression. Perhaps I am looking in the wrong place.
This would be very useful to allow a reduction in bandwidth for IoT devices with very small data tariffs.
Would it be possible to add this? I'm happy to have a go at writing a patch for this but would like to get some feedback from the maintainers first.
Thank you, Phil.
The text was updated successfully, but these errors were encountered: