Update types and handle data_base64 structured. #34

evankanderson · 2020-05-08T23:09:12Z

Add sane defaults for encoding
- Unfortunately, defaults for structured and binary need to be different
- Push types through interfaces
Make it easy to call 'ToRequest' using Marshaller defaults
Add tests for above
Resolves SDK must serialize/deserialize binary data to/from data_base64 instead of data #28 and slighly improves Consider a more Pythonic API #26
I discovered that the types for data_marshaller and data_unmarshaller had to vary between structured and unstructured as a user of the SDK. In particular, it seemed that UnmarshalJSON wants to pass a str value to data_unmarshaller, while UnmarshalBinary wants to pass a typing.IO.
I started by figuring out the necessary types for data_marshaller and data_unmarshaller. Along the way, I realized that binary CloudEvents data was not being handled correctly for JSON. See SDK must serialize/deserialize binary data to/from data_base64 instead of data #28 and https://github.com/cloudevents/spec/blob/v1.0/json-format.md#31-handling-of-data. Annoyingly, if the data is not of type "binary", the structured format suggests passing it through without an extra layer of enveloping, which leads to some special cases in MarshalJSON.
How to verify it

I added tests

One line description for the changelog

Update python types and handle structured `data_base64`.

Note that this is probably a breaking API change because the expected types for data_marshaller and data_unmarshaller changed and all previous clients needed to supply these.

- Add sane defaults for encoding - Unfortunately, defaults for structured and binary need to be *different* - Push types through interfaces - Make it easy to call 'ToRequest' using Marshaller defaults - Add tests for above Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

See https://gitlab.com/pycqa/flake8/-/issues/466 for details. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

evankanderson · 2020-05-13T20:46:36Z

I figured out how to fix the lint warnings; since it appears that the current configuration checks both W503 and W504, which are contradictory.

evankanderson · 2020-05-19T04:54:04Z

@denismakogon any interest in this?

evankanderson · 2020-05-29T17:36:38Z

Ping?

evankanderson · 2020-06-05T00:21:36Z

Ping 2

…

On Fri, May 29, 2020 at 10:36 AM Evan Anderson ***@***.***> wrote: Ping? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4XEN6U5QILJ6MNGANUPFTRT7XDHANCNFSM4M4QY57Q> .

grant · 2020-06-08T21:21:03Z

@evankanderson Who are you pinging?

evankanderson · 2020-06-08T23:55:32Z

I think @denismakogon is the only one who can approve/merge, but maybe @duglin can assist?

duglin · 2020-06-09T00:00:45Z

I can merge but I'd need at least one person who knows python to review... I don't know python

duglin · 2020-06-09T00:01:10Z

@di ^^^

di

LGTM, just a couple small things.

cloudevents/sdk/event/base.py

cloudevents/sdk/converters/structured.py

cloudevents/sdk/event/base.py

grant · 2020-06-22T16:29:10Z

Ping @evankanderson. It looks like you have a couple pending requested changes here.

evankanderson · 2020-06-25T21:41:43Z

@di Thanks for the previous review; ready for another look.

cloudevents/sdk/__init__.py

di

LGTM once DCO is passing (let me know if you need help with that)

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

cumason123 · 2020-06-30T15:33:19Z

Needs to update samples/http-cloudevents sample code

cumason123

For the most part everything looks good. I do have a couple design questions/nits however.

To my understanding, a consumer must first strip cloudevent fields of the ce- prefix before instantiating a CloudEvent. Could we perhaps do this field stripping internally to reduce overhead for consumers?

from flask import Flask, request
from cloudevents.sdk.http_events import CloudEvent
app = Flask(__name__)

@app.route("/", methods=["POST"])
def echo():
    attributes = {}
    
    contenttype = request.headers.get('Content-Type')
    if contenttype is not None:
        attributes['Content-Type'] = contenttype
        
    # unsure if passing None quietly fails thus I verify if field exists before
    # passing it into the CloudEvent   
    ce_type = request.headers.get('ce-type', request.form.get('type'))
    if ce_type is not None:
        # CloudEvent atm explicitly wants non ce- prefixed cloudevent fields
        attributes['type'] = ce_type
        
    ce_id = request.headers.get('ce-id', request.form.get('id'))
    if ce_id is not None:
        attributes['id'] = ce_id
        
    ce_source = request.headers.get('ce-source', request.form.get('source'))
    if ce_source is not None:
        attributes['source'] = ce_source
       
    ce_specversion = request.headers.get('ce-specversion', request.form.get('specversion'))
    if ce_specversion is not None:
        attributes['specversion'] = ce_specversion
    
    data = request.form.get('data_base64', request.form.get('data'))
    event = CloudEvent(attributes, data)
    
    # Do some work
    return '', 204

if __name__ == '__main__':
    app.run()

However this is how a consumer would ideally like to instantiate events:

from flask import Flask, request
from cloudevents.sdk.http_events import CloudEvent
app = Flask(__name__)

@app.route("/", methods=["POST"])
def home():
    headers = dict(request.form)
    body = dict(request.headers)
    
    # CloudEvent automatically parses the http headers/body
    event = CloudEvent(headers, body)
    
    # Do some work 
    return '', 204

if __name__ == '__main__':
    app.run()

cloudevents/sdk/http_events.py

cloudevents/tests/test_http_events.py

MarcBoissonneault

I love these additions/changes, keep up the good work!

MarcBoissonneault · 2020-06-30T16:24:48Z

cloudevents/sdk/http_events.py

+    def FromHttp(
+        cls,
+        data: typing.Union[str, bytes],
+        headers: dict = {},
+        data_unmarshaller: types.UnmarshallerType = None
    ):


Why do we decide to not follow the pep8 convention for naming of python methods:
pep8 says they should be:

def from_http(...) -> ...: ...

Also can we make sure to do "complete type hints":

Suggested change

def FromHttp(

cls,

data: typing.Union[str, bytes],

headers: dict = {},

data_unmarshaller: types.UnmarshallerType = None

):

def from_http(

cls,

data: typing.IO,

headers: typing.Dict[str, typing.Any],

data_unmarshaller: types.UnmarshallerType = None

) -> CloudEvent:

Technically, this is a class method, but point taken on naming, and I totally forgot to document the return type.

It's also incorrect to omit the HTTP headers, so removing the default is correct.

I'm not sure about the argument type hints:

typing.IO expects to have a read() method, but a quick survey of 3 popular Python libraries (flask, requests, sanic), suggests that the normal exposure of the request body is via a string where the library handles the read, rather than exposing the underlying socket.

I think the headers dict from HTTP clients would be a typing.Dict[str, str], not a [str, typing.Any], no?

Hmm -- it turns out that I can't use -> CloudEvent to document the return type, because the class is not defined until after all the functions have been defined (including this one).

Thoughts?

Hmm -- it turns out that I can't use -> CloudEvent to document the return type, because the class is not defined until after all the functions have been defined (including this one).

Thoughts?

for this add the following at the top:

from __future__ import annotations

For this:

typing.IO expects to have a read() method, but a quick survey of 3 popular Python libraries (flask, requests, sanic), suggests that the normal exposure of the request body is via a string where the library handles the read, rather than exposing the underlying socket.

I just used the exact same typehints that the current implementation is using here: https://github.com/cloudevents/sdk-python/blob/master/cloudevents/sdk/marshaller.py#L45

BUT after reading your comment, I think you are right!
And for the headers I also think you are right!

So, double 👍

It turns out that PEP 563 is only supported in Python 3.7 or later.

and that's exactly why the added this feature in the "future" module.

And this line in the imports:

from __future__ import annotations

And it should work.

It doesn't work on Python 3.6, which is in the test suite.

I tried adding a PR to do that, and the 3.6 tests (only) failed. If we can abandon 3.6, I'd be happy to make the change.

Python 3.6 won't be EOL until Dec 2021, so I think we should probably continue to support it.

cloudevents/sdk/http_events.py

MarcBoissonneault · 2020-06-30T16:40:56Z

tox.ini

@@ -17,6 +17,7 @@ commands =
    flake8

 [flake8]
-ignore = H405,H404,H403,H401,H306,S101,N802,N803,N806,I202,I201
+# See https://gitlab.com/pycqa/flake8/-/issues/466 for extend-ignore vs ignore
+extend-ignore = H405,H404,H403,H401,H306,S101,N802,N803,N806,I202,I201


All those "ignore" seems to indicate that we are not following standard python conventions ;)

https://flake8.pycqa.org/en/latest/user/error-codes.html

Not sure what these are; I discovered that extend-ignore was having issues with W503/W504, where flake8 changed the behavior.

H405, H404, H403, H401 are all about docstrings and spaces (I noticed some of these)

S101 - Multi-line missing trailing comma

N802, N803, N806 - function names lowercase, argument name lowercase, functiov in variable lowercase.

I201, I202 - Newlines between import groups

Maybe we can clean these up in a separate PR before 1.0.0?

FOR SURE we can change that in another PR!!!
It was just a comment about this whole repo where there seems to have lot of ignored rules...
Usually when we ignore rules it's because we don't like them or because we don't know the standard of the language.

Another thoughts here, personally I prefer pylint as linter (compare to flake8) since you can ignore rule by name instead of by "rule code" that are so hard (impossible) to remember.
For example, see this stackoverflow comment from 2014: https://stackoverflow.com/questions/4341746/how-do-i-disable-a-pylint-warning/23542817#23542817

I totally agree on the code not feeling pythonic -- the existing code feels very Java-ish to me.

With that said, coming from go-land, I hate linters because it introduces human debug-fix loops where presumably the machine could do the formatting for me.

I agree (these linting settings also predate me). I'd prefer if we just used https://pypi.org/project/black/, which includes an auto-formatter.

cumason123 · 2020-06-30T17:31:56Z

Can we adjust the sample code at samples/http-cloudevents, or should we do that in a separate PR?

evankanderson · 2020-07-02T20:21:12Z

For the most part everything looks good. I do have a couple design questions/nits however.

To my understanding, a consumer must first strip cloudevent fields of the ce- prefix before instantiating a CloudEvent. Could we perhaps do this field stripping internally to reduce overhead for consumers?

I think your understanding is colored by looking at the API through the lens of HTTP-Binary, with an assumed JSON payload. 😁 Ideally, the CloudEvents class would be both transport and payload independent, so you could send an "image/jpeg" event as a POST reply, or store a CloudEvent as an AVRO structured format in Kafka.

To instantiate a CloudEvent, the customer should provide the attribute names, rather than the HTTP-Binary header names. If you want to convert an HTTP-Binary request to a CloudEvent use CloudEvent.FromHttp(body, headers), which should take an un-decoded body and possibly a data_marshaller if the payload is something exotic, rather than JSON.

However this is how a consumer would ideally like to instantiate events:

from flask import Flask, request
from cloudevents.sdk.http_events import CloudEvent
app = Flask(__name__)

@app.route("/", methods=["POST"])
def home():
    headers = dict(request.form)
    body = dict(request.headers)
    
    # CloudEvent automatically parses the http headers/body
    event = CloudEvent(headers, body)
    
    # Do some work 
    return '', 204

if __name__ == '__main__':
    app.run()

I'd suggest the following for reading CloudEvents, which will work whether or not the CloudEvents payload is JSON:

from flask import Flask, request
from cloudevents.sdk.http_events import CloudEvent
app = Flask(__name__)

@app.route("/", methods=["POST"])
def home():
    # CloudEvent automatically parses the http headers/body, whether or not the body is JSON
    event = CloudEvent.FromHttp(request.get_data(), request.headers)

    # Do some work 
    return '', 204

if __name__ == '__main__':
    app.run()

evankanderson · 2020-07-02T21:41:54Z

Needs to update samples/http-cloudevents sample code

Any thoughts on whether/how we could add a test that the sample works?

evankanderson

Updated, PTAL!

cloudevents/sdk/http_events.py

cloudevents/tests/test_http_events.py

evankanderson · 2020-07-02T21:18:48Z

tox.ini

@@ -17,6 +17,7 @@ commands =
    flake8

 [flake8]
-ignore = H405,H404,H403,H401,H306,S101,N802,N803,N806,I202,I201
+# See https://gitlab.com/pycqa/flake8/-/issues/466 for extend-ignore vs ignore
+extend-ignore = H405,H404,H403,H401,H306,S101,N802,N803,N806,I202,I201


https://flake8.pycqa.org/en/latest/user/error-codes.html

Not sure what these are; I discovered that extend-ignore was having issues with W503/W504, where flake8 changed the behavior.

H405, H404, H403, H401 are all about docstrings and spaces (I noticed some of these)

S101 - Multi-line missing trailing comma

N802, N803, N806 - function names lowercase, argument name lowercase, functiov in variable lowercase.

I201, I202 - Newlines between import groups

Maybe we can clean these up in a separate PR before 1.0.0?

evankanderson · 2020-07-02T21:31:05Z

cloudevents/sdk/http_events.py

+    def FromHttp(
+        cls,
+        data: typing.Union[str, bytes],
+        headers: dict = {},
+        data_unmarshaller: types.UnmarshallerType = None
    ):


Hmm -- it turns out that I can't use -> CloudEvent to document the return type, because the class is not defined until after all the functions have been defined (including this one).

Thoughts?

…ell as JSON. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

…uman has to fix them. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Fix usability of binary detection in MarshalJSON to support memoryview. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

evankanderson · 2020-07-06T17:31:06Z

Hope everyone had a good 4th of July weekend!

I've updated the README.md; please take a look at the new API(s), since I've made breaking changes. I suspect that's okay given the pre-1.0 library and generally low adoption. If there are further changes we should make, it's probably better to make them sooner than later.

I've added a test for uploading bz2-encoded binary data and round-tripping that through the library. I didn't do a raw test for that because the content is opaque, but I can add one if it feels important.

samples/http-cloudevents/client.py

samples/http-cloudevents/server.py

cumason123 · 2020-07-06T19:52:46Z

cloudevents/sdk/http_events.py

+            data_unmarshaller = _json_or_string
+
+        event = marshaller.NewDefaultHTTPMarshaller().FromRequest(
+            v1.Event(), headers, data, data_unmarshaller)


Should we toggle between v1.Event() and v03.Event() depending on specversion?

Until we've parsed the event, it's hard to find the specversion. Fortunately, since we extract everything from the Event and throw it away, all we need are the parsing, which seems to be the same path for v3 and v1.

I agree that it's not ideal, but I think it will take a bunch of refactoring to separate the attribute extraction from the representation.

Alright. I'll make an issue for this once the PR gets merged to keep the diff count low.

cloudevents/tests/test_http_events.py

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

grant · 2020-07-07T15:48:36Z

Are we good to merge this @evankanderson @cumason123?

Out of date

cumason123

Possible refactor we can do either here or in a future PR:

...
from cloudevents.sdk import converters

headers, body = event.to_http(converters.TypeBinary)

could be refactored into:

headers, body = event.to_http(isbinary=True)

User no longer has to import converters. Functionally speaking, converters only has two possible string types anyways.

evankanderson · 2020-07-07T19:40:32Z

I'd prefer to keep the types for now.

My suspicion is that to_http and from_http are actually not the correct methods on a CloudEvent, because then we'll need to add to_kafka, to_amqp, etc.

I agree that it would be nice to have a better way to type event.to_http(converters.TypeBinary), but I think I might prefer to spell it http.binary.emit(event) or http.structured.parse(headers, body).

evankanderson · 2020-07-08T21:49:33Z

It looks like there are approvals and no requested changes. @cumason123 -- are you comfortable with this being committed?

* Update types and handle data_base64 structured. - Add sane defaults for encoding - Unfortunately, defaults for structured and binary need to be *different* - Push types through interfaces - Make it easy to call 'ToRequest' using Marshaller defaults - Add tests for above Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix lint warnings due to changes to W503/W504 See https://gitlab.com/pycqa/flake8/-/issues/466 for details. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Adopt di's suggestions. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix lint. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Move types to another package. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Adjust CloudEvent class in http_events.py to support binary data as well as JSON. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Apply suggested changes by MacrBoissonneault Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix samples as well. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix lint. Apparently, we can complain about formating issues, but a human has to fix them. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Add test for binary encoding of messages. Fix usability of binary detection in MarshalJSON to support memoryview. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix errors noticed by cumason123 Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> Signed-off-by: Curtis Mason <cumason@google.com>

* Created CloudEvent class (#36) CloudEvents is a more pythonic interface for using cloud events. It is powered by internal marshallers and cloud event base classes. It performs basic validation on fields, and cloud event type checking. Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * Implemented python properties in base.py (#41) * Added SetCloudEventVersion Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * began adding python properties Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * added pythonic properties to base class Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * began testing for getters/setters Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * added general setter tests Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * fixed spacing in base.py Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * added __eq__ to option and datacontentencoding property to v03 Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * lint fixes Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * testing extensions and old getters Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * removed versions v01 and v02 from test_data_encaps_refs.py Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * fixed inheritance issue in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * added prefixed_headers dict to test Signed-off-by: Curtis Mason <cumason@google.com> * Http structured cloudevents (#47) * Moved fields out of base & structured support base._ce_required_fields and base._ce_optional_fields were moved into event classes v03 and v1. http_events.CloudEvent class now looks for fieldnames in either headers or data, and can automatically determine whether this is a binary or structured event. Signed-off-by: Curtis Mason <cumason@google.com> * testing structured Signed-off-by: Curtis Mason <cumason@google.com> * added tests for structured events Signed-off-by: Curtis Mason <cumason@google.com> * Added test valid structured cloudevents Signed-off-by: Curtis Mason <cumason@google.com> * Created default headers arg in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * Added http_events.py sample code Signed-off-by: Curtis Mason <cumason@google.com> * removed ../python-event-requests Signed-off-by: Curtis Mason <cumason@google.com> * README.md nit Signed-off-by: Curtis Mason <cumason@google.com> * client.py nit Signed-off-by: Curtis Mason <cumason@google.com> * comment nits Signed-off-by: Curtis Mason <cumason@google.com> * created __getitem__ in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * sample nits Signed-off-by: Curtis Mason <cumason@google.com> * fixed structured empty data issue Signed-off-by: Curtis Mason <cumason@google.com> * Added CloudEvent to README Signed-off-by: Curtis Mason <cumason@google.com> * added http_msg to CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * implemented ToRequest in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * testing more specversions Signed-off-by: Curtis Mason <cumason@google.com> * Added sample code to README.md Signed-off-by: Curtis Mason <cumason@google.com> * modified sample code Signed-off-by: Curtis Mason <cumason@google.com> * added datavalidation to changelog Signed-off-by: Curtis Mason <cumason@google.com> * updated README Signed-off-by: Curtis Mason <cumason@google.com> * README adjustment Signed-off-by: Curtis Mason <cumason@google.com> * ruler 80 adjustment on http_events Signed-off-by: Curtis Mason <cumason@google.com> * style and renamed ToRequest to to_request Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * fixed self.binary typo Signed-off-by: Curtis Mason <cumason@google.com> * CHANGELOG adjustment Signed-off-by: Curtis Mason <cumason@google.com> * rollback CHANGELOG Signed-off-by: Curtis Mason <cumason@google.com> * Added documentation to to_request Signed-off-by: Curtis Mason <cumason@google.com> * README.md adjustment Signed-off-by: Curtis Mason <cumason@google.com> * renamed event_handler to event_version Signed-off-by: Curtis Mason <cumason@google.com> * inlined field_name_modifier Signed-off-by: Curtis Mason <cumason@google.com> * renamed test body data Signed-off-by: Curtis Mason <cumason@google.com> * removed unnecessary headers from test Signed-off-by: Curtis Mason <cumason@google.com> * removed field_name_modifier and fixed e.g. in client.py Signed-off-by: Curtis Mason <cumason@google.com> * pylint fix Signed-off-by: Curtis Mason <cumason@google.com> * Update types and handle data_base64 structured. (#34) * Update types and handle data_base64 structured. - Add sane defaults for encoding - Unfortunately, defaults for structured and binary need to be *different* - Push types through interfaces - Make it easy to call 'ToRequest' using Marshaller defaults - Add tests for above Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix lint warnings due to changes to W503/W504 See https://gitlab.com/pycqa/flake8/-/issues/466 for details. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Adopt di's suggestions. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix lint. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Move types to another package. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Adjust CloudEvent class in http_events.py to support binary data as well as JSON. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Apply suggested changes by MacrBoissonneault Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix samples as well. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix lint. Apparently, we can complain about formating issues, but a human has to fix them. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Add test for binary encoding of messages. Fix usability of binary detection in MarshalJSON to support memoryview. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Fix errors noticed by cumason123 Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com> * Changelog version deprecation (#48) * added changelog Signed-off-by: Curtis Mason <cumason@google.com> * Created CloudEvent class (#36) CloudEvents is a more pythonic interface for using cloud events. It is powered by internal marshallers and cloud event base classes. It performs basic validation on fields, and cloud event type checking. Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Fix tox configuration for CI (#46) Signed-off-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Implemented python properties in base.py (#41) * Added SetCloudEventVersion Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * began adding python properties Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * added pythonic properties to base class Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * began testing for getters/setters Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * added general setter tests Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * fixed spacing in base.py Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * added __eq__ to option and datacontentencoding property to v03 Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * lint fixes Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * testing extensions and old getters Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * removed versions v01 and v02 from test_data_encaps_refs.py Signed-off-by: Curtis Mason <cumason@google.com> Signed-off-by: Dustin Ingram <di@users.noreply.github.com> * fixed inheritance issue in CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * added prefixed_headers dict to test Signed-off-by: Curtis Mason <cumason@google.com> * CHANGELOG adjustment Signed-off-by: Curtis Mason <cumason@google.com> * Update CHANGELOG.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Update CHANGELOG.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Update CHANGELOG.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Update CHANGELOG.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Removed irrelevant files from commit diff Signed-off-by: Curtis Mason <cumason@google.com> Co-authored-by: Dustin Ingram <di@users.noreply.github.com> * Black formatter (#51) * black and isort added to precommit Signed-off-by: Curtis Mason <cumason@google.com> * main renaming Signed-off-by: Curtis Mason <cumason@google.com> * fixed tox Signed-off-by: Curtis Mason <cumason@google.com> * linting in tox rename Signed-off-by: Curtis Mason <cumason@google.com> * fixed tox trailing space Signed-off-by: Curtis Mason <cumason@google.com> * added reformat tox env Signed-off-by: Curtis Mason <cumason@google.com> * Reformatting files Signed-off-by: Curtis Mason <cumason@google.com> * reformatted more files Signed-off-by: Curtis Mason <cumason@google.com> * documented tox in README Signed-off-by: Curtis Mason <cumason@google.com> * removed -rc flag Signed-off-by: Curtis Mason <cumason@google.com> * README and http-cloudevents sample code adjustments to reflect new CloudEvent (#56) * README and http-cloudevents CloudEvent adjustments README no longer shows how to use base event classes to create events. Removed this because users shouldn't be forced to interact with the marshaller class. Additionally, CloudEvent is a simpler interface therefore we are encouraging the CloudEvent class usage. http-cloudevents now has more example usage for the getitem overload. Similarly README shows how to use getitem overload. Signed-off-by: Curtis Mason <cumason@google.com> * lint reformat Signed-off-by: Curtis Mason <cumason@google.com> * resolved nits Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * renamed /mycontext to url Signed-off-by: Curtis Mason <cumason@google.com> * renamed here linlk to in the samples directory Signed-off-by: Curtis Mason <cumason@google.com> * Separated http methods (#60) * instantiated http path Signed-off-by: Curtis Mason <cumason@google.com> * moved from_http from CloudEvent to http Signed-off-by: Curtis Mason <cumason@google.com> * Moved to_http out of CloudEvent Signed-off-by: Curtis Mason <cumason@google.com> * moved http library into event.py Signed-off-by: Curtis Mason <cumason@google.com> * testing printable cloudevent Signed-off-by: Curtis Mason <cumason@google.com> * Adjusted README Signed-off-by: Curtis Mason <cumason@google.com> * Created EventClass Signed-off-by: Curtis Mason <cumason@google.com> * reformatted event.py Signed-off-by: Curtis Mason <cumason@google.com> * from_json definition Signed-off-by: Curtis Mason <cumason@google.com> * server print changes Signed-off-by: Curtis Mason <cumason@google.com> * Specversion toggling (#57) * cloudevent now switches specversion types Signed-off-by: Curtis Mason <cumason@google.com> * removed duplicate marshall instance Signed-off-by: Curtis Mason <cumason@google.com> * resolved grant requests Signed-off-by: Curtis Mason <cumason@google.com> * converters now can check headers for fields Signed-off-by: Curtis Mason <cumason@google.com> * removed print statement Signed-off-by: Curtis Mason <cumason@google.com> * Fixed marshallers looking at headers for specversion Signed-off-by: Curtis Mason <cumason@google.com> * lint fixes Signed-off-by: Curtis Mason <cumason@google.com> * is_binary static method and structured isinstance rework Signed-off-by: Curtis Mason <cumason@google.com> * testing for is_binary and is_structured Signed-off-by: Curtis Mason <cumason@google.com> * Image sample code (#65) * added image example Signed-off-by: Curtis Mason <cumason@google.com> * moved size into headers Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * renamed sample code Signed-off-by: Curtis Mason <cumason@google.com> * added test to http-image-cloudevents sample Signed-off-by: Curtis Mason <cumason@google.com> * removed unnecessary function Signed-off-by: Curtis Mason <cumason@google.com> * Added testing for http-image-cloudevents Signed-off-by: Curtis Mason <cumason@google.com> * Data marshall arg fix and better image in sample Fixed bug where data_marshaller and data_unmarshaller wasn't being passed into positional arguments. Also used cloudevents logo for the image in http-image-cloudevents Signed-off-by: Curtis Mason <cumason@google.com> * adjusted http-image-cloudevents samples Signed-off-by: Curtis Mason <cumason@google.com> * reformat and README changes Signed-off-by: Curtis Mason <cumason@google.com> * io bytes casting in data_unmarshaller Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * removed unusued imports in http-image samples Signed-off-by: Curtis Mason <cumason@google.com> * removed samples/http-cloudevents/tmp.png Signed-off-by: Curtis Mason <cumason@google.com> * Nits Signed-off-by: Curtis Mason <cumason@google.com> * Implemented to_json and from_json (#72) * added test_to_json test Signed-off-by: Curtis Mason <cumason@google.com> * implemented to_json with tests Signed-off-by: Curtis Mason <cumason@google.com> * from_json and to_json tests Signed-off-by: Curtis Mason <cumason@google.com> * Tests for to_json being able to talk to from_json Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * added documentation for to_json and from_json Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * Fixed top level extensions bug (#71) * Fixed top level extensions bug Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * fixed name bug in test_event_extensions Signed-off-by: Curtis Mason <cumason@google.com> * fixed broken links in README.md (#75) Signed-off-by: Curtis Mason <cumason@google.com> * Fixed marshaller documentation typo's in http (#76) * Fixed marshaller documentation in http directory Signed-off-by: Curtis Mason <cumason@google.com> * adjusted marshaller documentation Signed-off-by: Curtis Mason <cumason@google.com> * None data fix (#78) * fixed none data issue Signed-off-by: Curtis Mason <cumason@google.com> * added none data test for marshalling Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * Samples image test server (#79) * fixed none data issue Signed-off-by: Curtis Mason <cumason@google.com> * added none data test for marshalling Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * added http server test in image sample Signed-off-by: Curtis Mason <cumason@google.com> * Removed print statements from test Signed-off-by: Curtis Mason <cumason@google.com> * removed requests from test Signed-off-by: Curtis Mason <cumason@google.com> * Top level http (#83) * Modularized http and made http a top level module Modularized the http directory by separating related functions into different scripts. Also removed EventClass and kept a singular CloudEvent. Finally, CloudEvent.__repr__ was refactored such that it doesn't depend on external methods. Signed-off-by: Curtis Mason <cumason@google.com> * renamed requests.py to http_methods Signed-off-by: Curtis Mason <cumason@google.com> * lint fixes Signed-off-by: Curtis Mason <cumason@google.com> * http-json-cloudevents testing (#80) * Added tests to http-json-cloudevents Signed-off-by: Curtis Mason <cumason@google.com> * removed outdated python-requests sample code Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * Added flask to requirements Signed-off-by: Curtis Mason <cumason@google.com> * lint fix Signed-off-by: Curtis Mason <cumason@google.com> * docs: add README badge (#85) Signed-off-by: Grant Timmerman <timmerman+devrel@google.com> * added pypi-release rule (#87) * added pypi-release rule Signed-off-by: Curtis Mason <cumason@google.com> * added RELEASING.md Signed-off-by: Curtis Mason <cumason@google.com> * Adjusted RELEASING.md Signed-off-by: Curtis Mason <cumason@google.com> * Update .github/workflows/pypi-release.yml Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * workflow pypi name changed Signed-off-by: Curtis Mason <cumason@google.com> * Update RELEASING.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Update RELEASING.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * Update RELEASING.md Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * removed some pbr stuff Signed-off-by: Curtis Mason <cumason@google.com> * Removed all pbr stuff Signed-off-by: Curtis Mason <cumason@google.com> * README nits Signed-off-by: Curtis Mason <cumason@google.com> * RELEASING adjustment in README Signed-off-by: Curtis Mason <cumason@google.com> * author update in setup.cfg Signed-off-by: Curtis Mason <cumason@google.com> * removed setup.cfg Signed-off-by: Curtis Mason <cumason@bu.edu> * Update setup.py Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@bu.edu> * lint fix Signed-off-by: Curtis Mason <cumason@bu.edu> Co-authored-by: Dustin Ingram <di@users.noreply.github.com> * pypi-release git tags automation (#88) * added pypi_packaging Signed-off-by: Curtis Mason <cumason@bu.edu> * reverted pypi-release Signed-off-by: Curtis Mason <cumason@bu.edu> * added pypi_package workflow Signed-off-by: Curtis Mason <cumason@bu.edu> * added gitpython dependency Signed-off-by: Curtis Mason <cumason@bu.edu> * added git import in createTag function Signed-off-by: Curtis Mason <cumason@bu.edu> * Updated RELEASING.md and implemented pypi_config in pypi_packaging.pg Signed-off-by: Curtis Mason <cumason@bu.edu> Signed-off-by: Curtis Mason <cumason@google.com> * Fixed some docs Signed-off-by: Curtis Mason <cumason@bu.edu> Signed-off-by: Curtis Mason <cumason@google.com> * Update .github/workflows/pypi-release.yml Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Signed-off-by: Curtis Mason <cumason@google.com> * added __version__ Signed-off-by: Curtis Mason <cumason@google.com> * lint change Signed-off-by: Curtis Mason <cumason@google.com> * reinstalling cloudevents in workflow Signed-off-by: Curtis Mason <cumason@google.com> * added cloudevents to publish.txt Signed-off-by: Curtis Mason <cumason@google.com> * removed old release_doc Signed-off-by: Curtis Mason <cumason@google.com> Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Co-authored-by: Evan Anderson <evan.k.anderson@gmail.com> Co-authored-by: Dustin Ingram <di@users.noreply.github.com> Co-authored-by: Grant Timmerman <timmerman@google.com>

evankanderson force-pushed the simplify-args branch from 0ac297c to c1600cb Compare May 8, 2020 23:14

evankanderson force-pushed the simplify-args branch from c1600cb to 4ae305d Compare May 8, 2020 23:21

Fix lint warnings due to changes to W503/W504

2f96d28

See https://gitlab.com/pycqa/flake8/-/issues/466 for details. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

evankanderson force-pushed the simplify-args branch from 1445266 to 2f96d28 Compare May 13, 2020 20:46

grant approved these changes May 21, 2020

View reviewed changes

di self-requested a review June 9, 2020 15:58

di reviewed Jun 11, 2020

View reviewed changes

cloudevents/sdk/event/base.py Outdated Show resolved Hide resolved

cloudevents/sdk/converters/structured.py Outdated Show resolved Hide resolved

cloudevents/sdk/event/base.py Outdated Show resolved Hide resolved

di changed the base branch from master to v1.0.0-dev June 11, 2020 16:31

grant assigned evankanderson Jun 15, 2020

Merge remote-tracking branch 'upstream/master' into simplify-args

3d3be3c

evankanderson force-pushed the simplify-args branch 2 times, most recently from 600e688 to ddfc749 Compare June 25, 2020 21:48

di reviewed Jun 25, 2020

View reviewed changes

cloudevents/sdk/__init__.py Outdated Show resolved Hide resolved

di previously approved these changes Jun 25, 2020

View reviewed changes

evankanderson added 4 commits June 26, 2020 12:05

Adopt di's suggestions.

84e5f2a

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Merge remote-tracking branch 'upstream/v1.0.0-dev' into simplify-args

4f8fb6f

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Fix lint.

b99a138

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Move types to another package.

bb8498d

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

evankanderson force-pushed the simplify-args branch from 51d0d41 to bb8498d Compare June 26, 2020 19:15

cumason123 requested changes Jun 30, 2020

View reviewed changes

MarcBoissonneault reviewed Jun 30, 2020

View reviewed changes

evankanderson commented Jul 2, 2020

View reviewed changes

evankanderson force-pushed the simplify-args branch 2 times, most recently from c079493 to 4a45198 Compare July 2, 2020 22:12

evankanderson added 4 commits July 2, 2020 15:17

Adjust CloudEvent class in http_events.py to support binary data as w…

e0f2d09

…ell as JSON. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Apply suggested changes by MacrBoissonneault

a900510

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Fix samples as well.

9892427

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

Fix lint. Apparently, we can complain about formating issues, but a h…

e4f2d2f

…uman has to fix them. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

evankanderson force-pushed the simplify-args branch 2 times, most recently from 748b896 to e4f2d2f Compare July 3, 2020 04:47

Add test for binary encoding of messages.

99b3db4

Fix usability of binary detection in MarshalJSON to support memoryview. Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

cumason123 requested changes Jul 6, 2020

View reviewed changes

Fix errors noticed by cumason123

3be7885

Signed-off-by: Evan Anderson <evan.k.anderson@gmail.com>

grant requested a review from cumason123 July 7, 2020 15:49

cumason123 approved these changes Jul 7, 2020

View reviewed changes

cumason123 self-requested a review July 7, 2020 16:00

cumason123 reviewed Jul 7, 2020

View reviewed changes

cumason123 merged commit 7e054fa into cloudevents:v1.0.0-dev Jul 8, 2020

Update types and handle data_base64 structured. #34

Update types and handle data_base64 structured. #34

Conversation

evankanderson commented May 8, 2020

evankanderson commented May 13, 2020

evankanderson commented May 19, 2020

evankanderson commented May 29, 2020

evankanderson commented Jun 5, 2020 via email

grant commented Jun 8, 2020

evankanderson commented Jun 8, 2020

duglin commented Jun 9, 2020

duglin commented Jun 9, 2020

di left a comment

Choose a reason for hiding this comment

grant commented Jun 22, 2020

evankanderson commented Jun 25, 2020

di left a comment

Choose a reason for hiding this comment

cumason123 commented Jun 30, 2020

cumason123 left a comment • edited Loading

Choose a reason for hiding this comment

MarcBoissonneault left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarcBoissonneault Jul 2, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cumason123 commented Jun 30, 2020

evankanderson commented Jul 2, 2020 • edited Loading

evankanderson commented Jul 2, 2020

evankanderson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

evankanderson commented Jul 6, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cumason123 Jul 7, 2020 • edited Loading

Choose a reason for hiding this comment

grant commented Jul 7, 2020

cumason123 left a comment • edited Loading

Choose a reason for hiding this comment

evankanderson commented Jul 7, 2020

evankanderson commented Jul 8, 2020

cumason123 left a comment •

edited

Loading

MarcBoissonneault Jul 2, 2020 •

edited

Loading

evankanderson commented Jul 2, 2020 •

edited

Loading

cumason123 Jul 7, 2020 •

edited

Loading

cumason123 left a comment •

edited

Loading