Thanks for the work.
I'll try to look at this during the week. The general trend on the project is to keep thing as clear and straight forward as possible.
I'll try to see if we can make those optimizations easy to plug if one needs / wants them.

The main speedup comes from 1) ability to use unsorted output (unfortunately Sorted/OrderedDicts are slow), and 2) skipping is_simple_callable() checks. The latter is harder to support in a clean and completely safe way.

Echoing @xordoquy here, yup. Almost certainly not a trade-off we'd consider, but certainly worth investigating anything that can make it simpler to do this as a third party serializer base class.

To expand on this slightly...

• In any situation where the data is backed by a data store of any kind the performance increases certainly won't be anything like as they appear here.
• If performance is critical I would rather take the approach of keeping the base classes as simple as possible so that it's in turn easier for the dev to override. In this type of case I'd advocate that what we should instead be doing is showing the developers via good documentation how they can override the entire to_representation method explicitly in order to bypass the normal machinery.

Eg...

class CommentSerializer(serializers.Serializer):
    email = serializers.EmailField()
    content1 = serializers.CharField(max_length=200)
    content2 = serializers.CharField(max_length=200)
    content3 = serializers.CharField(max_length=200)
    content4 = serializers.CharField(max_length=200)
    content5 = serializers.CharField(max_length=200)
    content6 = serializers.CharField(max_length=200)
    content7 = serializers.CharField(max_length=200)
    content8 = serializers.CharField(max_length=200)
    date = serializers.DateField()

    def to_representation(self, obj):
        """
        Serialization speed of these objects is critical, so we're hardcoding this...
        """
        return {
            'email': obj.email,
            'content1': obj.content1,
            ...
            'date': obj.date.isoformat()
        }

We'd also want to document up the advanced usage of how and when you'd want to use a sorted dict,, or the ReturnDict class (Allows you to support renderers that require backlinks to extra field information. When used it turns on HTML forms in the browsable API.)

Closing. Very much appreciate the input, but not quite in the direction I'd like us to take things for the reasons outlined above. Don't take that as meaning that we can't discuss this or similar ideas further, but keeping scope and direction on-track in the meantime.

@tomchristie Is there a way we can keep track of this work ? I'd like to keep it on our radars.

@xordoquy Happy for us to open a documentation ticket as detailed above, or role it into #1976.

Anything else depends on where @akaariai wants to take this. If there are API hooks that we ought to be adding into the serializer class in order to support these sorts of modifications then we can consider those on a case-by-case basis.

@akaariai Apologies also for my directness wrt to closing tickets off, not at all any comment on the worthiness on the patch, I've just been maintaining the project long enough now I think we've got a good feel for what helps and what harms the long-term maintenance and end-user usability of the project, and I'm very strongly in favor of us optimising for simplest-possible wherever possible.

My work was mainly trying to check where DRF uses extra time that could be easy to shave off. So, +1 to closing this :)

While the benchmark presented wasn't too realistic, I currently have a need for a faster serializer than what DRF can offer. When just reading objects from DB and serializing them the Serializer class is a bottleneck even in version-3.0. Of course, we aren't talking about huge changes here

Another idea could be adding a couple of built-in to_representation functions (or just documenting them).

# Add to Serializer:
    def obj_to_representation(self, obj):
          return dict([(f.field_name, f.to_representation(getattr(obj, f.field_name)))
                            for f in self.fields.values()])

    def dict_to_representation(self, d):
          return dict([(f.field_name, f.to_representation(d.get(f.field_name)))
                            for f in self.fields.values()])

# Now, a faster PeopleSerializer:

class PeopleSerializer(Serializer):
    to_representation = Serializer.obj_to_representation

Then, document how to write a faster to_representation() method, and point out that DRF includes these two convenience methods you can use directly. As a bonus these methods gives a runtime of 1.9s compared to 3.1s of this PR's version (or almost 5x faster than version-3.0). See akaariai@b2336f0 for this idea.

Strongly in favour of documenting something like this...

def obj_to_representation(self, obj):
      return dict([(f.field_name, f.to_representation(getattr(obj, f.field_name)))
                        for f in self.fields.values()])

Not sure about pulling the source into the package tho - rather call it out very explicitly as an example, but force folks to deal with any of the subtleties of the limitations it introduces themselves. Pulling it into core we'd almost certainly get 'but it doesn't work in XYZ case' etc...

This would also be a good conversation to have around where we introduce BaseSerializer...

https://github.com/tomchristie/django-rest-framework/blob/version-3.0/docs/topics/3.0-announcement.md#the-baseserializer-class

Noting that you don't always need to be using BaseSerializer in order to get big performance improvements.

Reopening this ticket, and re-naming as documentation-focused.

Oh wait, not correct, given that this was a PR. I'll open as a seperate issue instead. Sorry for the noise.

I implemented quickly a way to use non-ordered dicts, see akaariai@2ee4cdc. There shouldn't be any cave-eats with the approach except of course your output will be non-ordered. See the commit message for how to use the approach.

If this approach seems like something you'd like to support, then I am happy to open a separate pull request for that. This could also be implemented as a mixin outside DRF if the _dict_class attribute is available.

BTW If this PR isn't appropriate place for discussing further ideas I am happy to continue this discussion somewhere else. Also, thanks for your fast and informative responses to this PR!

I'm also seeing slowness issue with OrderedDict (#2454), but related to pickling. I like the customizable _dict_class in akaariai@2ee4cdc; I wonder if it might be a good idea to allow a global setting (in settings.REST_FRAMEWORK) as well

I wonder if it might be a good idea to allow a global setting (in settings.REST_FRAMEWORK) as well

I wouldn't currently be in a favor of that. For the large majority of cases DB access is going to be the limiting factor, so I'd prefer for this to just be an advanced bit of API limited to the serializers.

I think this is still a good thing to do (customizable _dict_class as in akaariai@2ee4cdc).

I'll try to get an updated patch (off of DRF 3.1), submit PR and maybe include some performance tests