Skip to content

[RFC] ObjectMapper #54476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Korbeil opened this issue Apr 3, 2024 · 25 comments
Open

[RFC] ObjectMapper #54476

Korbeil opened this issue Apr 3, 2024 · 25 comments
Labels
RFC RFC = Request For Comments (proposals about features that you want to be discussed)

Comments

@Korbeil
Copy link
Contributor

Korbeil commented Apr 3, 2024

Hey everyone 👋

This issue is here to standardize how we should think about mapping in future and a proposition on how we could harmonize all theses ideas in one component.

Context

Today we have some PR that are about mapping and we all have ideas on how we should do things. And we probably also already have use-case for all those ideas.

Currently when you think about mapping (object to object mapping, atleast), you will require to use the Serializer not exactly as it was designed. You'll need to normalize your source object then denormalize to the targeted object.

This is a bit complex for what we want to do and it brings some limitations:

  • We will have an intermediate array state, the bigger the source object is, the bigger the intermediate array will be;
  • If we want to customize some part of that mapping, we will need to write a Normalizer/Denormalizer for the whole transformation.

Due to these limitations, we think that there is a need for a new component that will map objects.

Current implementations

Let's list the current two main implementations:

Both theses implementations are trying to tackle object to object mapping. Indeed we have some different features because of how we implemented things:

  • @soyuka's PR focus on DX & simple implementation. The code is really easy to understand and it will do object to object mapping really well. It's mostly based onto PropertyAccessor component.
  • jolicode/automapper is focused on performance and can map a bit more than from an object to an object. It's more complex due to AST usage and is mostly based on PropertyInfo component (which will switch on TypeInfo once 7.1 is released)

Future

We talked with some core contributors during SymfonyLive Paris 2024 (28/29 march) and with @nicolas-grekas we came to the conclusion that we won't find a perfect implementation to merge as it is and we should iterate on all theses ideas.

In my opinion, the component should be focused on one main interface:

interface ObjectMapperInterface 
{
    /**
     * @template T
     * @template K
     * 
     * @param array|\stdClass|T $source
     * @param class-string<K>|'array'|K $target
     * @return array|K
     */
    public function map(array|object $source, string|object $target, array $context = []): array|object;
}

This interface will handle object to object mapping and array to object (or inverse) mapping.
I added array to object mapping (or inverse) because I do think we should it for some cases like handling a Request query parameters of to handle mapping Form values to an object.

For now, best thing would be to merge @soyuka's PR and iterate from that point towards theses interfaces with both implementations. Having two implementations isn't a big issue, we already have components working that way like DependencyInjection or ExpressionLanguage. Here the goal is to have two way to map objects:

  • Using reflection
  • Generating code

Using reflection makes the code way simpler and generating code is more complex but gives amazing performance.
Once the ObjectMapper PR is merged we need to make a wide and very good test suite so we can start adding code generating and validate we have same behavior with both implementations.

That way we can use any implementation whenever one fit better than the other for our use-case.

Metadata

Here is a section about metadata management within the ObjectMapper, I'll try to explain how metadata retrieval should work. There is 3 ways to collect metadata: from the source, from the target or from both.

We will fetch metadata from the source or from the target only when the source or target is either an array or a \stdClass. We will always fetch metadata from the non-array (or stdClass) object and try to match similar fields within the array or stdClass. When mapping to an array or stdClass, it's easy since all the fields are to create so there won't be matching issues. But when mapping from an array or stdClass, we will need to check if the fields does match properties in the target object, when it's matching we will map but if it's not matching we won't map that field.

Mapping an array to array (or stdClass) should throw an exception and is not possible.

Then, last method is getting metadata from both the source and target objects:
When doing that we will fetch all properties to map from both objects and if one of the source object property matches one of the target property, we will map that field. A property can match another property if its name match in both objects (or same configured name, if MapTo/MapFrom brings some new name).

Conclusion

I really do think ObjectMapper is an awesome component for Symfony. Today it's complex to improve the Serializer and I do think the "next" Serializer will be coming from another component or another set of component.
With the RFC as it is, we could replace the Normalization part of the Serializer.

In addition to component like @mtarld's JsonEncoder it could make a complete serialization replacement like Mathias & I showed at SymfonyLive 2024 with the TurboSerializer.

@carsonbot carsonbot added the RFC RFC = Request For Comments (proposals about features that you want to be discussed) label Apr 3, 2024
@stof
Copy link
Member

stof commented Apr 3, 2024

I'm wondering about the new array support.
A big point of the talk your talk during the SymfonyLive was that having good metadata was key in achieving a better mapping. array gives us no metadata at all about the structure inside that array.

And I'm wondering the same about the case of \stdClass in $source

@Korbeil
Copy link
Contributor Author

Korbeil commented Apr 3, 2024

We always need metadata from somewhere, when you map array or stdClass we get it from source or target (depending if you want an array as target or source) and you consider you array will match these metadata.

So for example if I do:

$mapper->map(['name' => 'stof'], Person::class);

If person has a name field, it will map cleanly, if not, nothing will be mapped (but you can use a MapFrom attribute to change the source field name)

@stof
Copy link
Member

stof commented Apr 3, 2024

I think this RFc would then a little bit of explanation about the source of metadata then. Unlike serializer which involves the metadata of a single class (which is then logically the source of truth for loading the metadata), the signature of the ObjectMapper means we have 2 types involved with different roles (from A to B). This makes it less clear how to decide where the metadata comes from (as the metadata for from A to B can be different than from A to C or from D to B so neither A alone nor B alone is enough to identify the metadata AFAICT).
And if we imagine that the metadata is defined in attributes in A (MapTo) and B (MapFrom), things might become confusing if both classes have those attributes (which one wins, or how do we combine them ?)

@Korbeil
Copy link
Contributor Author

Korbeil commented Apr 3, 2024

@stof after your last comment I tried to explain how metadata retrieval should work, here is what I added in this RFC:

Metadata

I'll try to explain how metadata retrieval should work. There is 3 ways to collect metadata: from the source, from the target or from both.

We will fetch metadata from the source or from the target only when the source or target is either an array or a \stdClass. We will always fetch metadata from the non-array (or stdClass) object and try to match similar fields within the array or stdClass. When mapping to an array or stdClass, it's easy since all the fields are to create so there won't be matching issues. But when mapping from an array or stdClass, we will need to check if the fields does match properties in the target object, when it's matching we will map but if it's not matching we won't map that field.

Mapping an array to array (or stdClass) should throw an exception and is not possible.

Then, last method is getting metadata from both the source and target objects:
When doing that we will fetch all properties to map from both objects and if one of the source object property matches one of the target property, we will map that field. A property can match another property if its name match in both objects (or same configured name, if MapTo/MapFrom brings some new name).

@yguedidi
Copy link
Contributor

yguedidi commented Apr 4, 2024

Mapping an array to array (or stdClass) should throw an exception and is not possible.

to be more explicit instead of an exception what about having 2 methods (potentially 2 interfaces for each then), being mapToArray and mapToObject (or just make map be the one mapping to object only, from object or array)?

@Korbeil
Copy link
Contributor Author

Korbeil commented Apr 4, 2024

@yguedidi it seems like too much for me to add 2 new interfaces to handle this, while with a simple exception and a clean message we can warn users and make a single interface

@maxbeckers
Copy link
Contributor

This sounds for me a bit like a Mapstruct in the java context to map data from one object to another one. Yes, i think so too, that we should also think about arrays.
I guess I would prefer the code generation way to keep up the performance instead of reflection.

In mapstruct you define an interface with the mapping methods and you can annotate on the methods some special mappings where it's not possible to map the attribute 1:1 to the other class.

Examplecode:

#[Mapping(target = "ignoreMe", ignore = true)]
#[Mapping(target = "mappMe", source = "fromHere")]
public function map (ClassA classA): ClassB

Mapstruct also allows to add expressions (e.g. call another function) or to define a specific named method that should be called (e.g. return null when the object is null or copy the object else). And it tries to find Methods for the sub-classes to map them, when it's an object and not just a string,int,... to map.

@Korbeil
Copy link
Contributor Author

Korbeil commented Apr 4, 2024

@maxbeckers for https://github.com/jolicode/automapper we were mostly inspired by https://github.com/AutoMapper/AutoMapper and I didn't know about MapStruct, but it looks really close indeed !

@maxbeckers
Copy link
Contributor

maxbeckers commented Apr 4, 2024

@Korbeil but if i read correctly in the docs, it is more a converter, that creates a new object of target class ... sth like this is not possible:

        Source source = new Source();
        source.setName("John");
        source.setAge(30);
        
        Target target = new Target();
        target.setCompany("Testcompany");
        
        target = mapper.mapSourceToTarget(source, target);
        // or just mapper.mapSourceToTarget(source, target); without `target = `
        
        System.out.println("Name in target object: " + target.getName()); // John
        System.out.println("Age in target object: " + target.getAge()); // 30
        System.out.println("Company in target object: " + target.getCompany()); // Testcompany

So with the automapper implementation, it's always a new Object that gets returned, correctly?

@joelwurtz
Copy link
Contributor

joelwurtz commented Apr 4, 2024

So with the automapper implementation, it's always a new Object that gets returned, correctly?

Not necessarily, if you pass an existing object in the target (instead of a classname) it will use it instead of creating a new one

@WeTurkstra
Copy link

I think this RFc would then a little bit of explanation about the source of metadata then. Unlike serializer which involves the metadata of a single class (which is then logically the source of truth for loading the metadata), the signature of the ObjectMapper means we have 2 types involved with different roles (from A to B). This makes it less clear how to decide where the metadata comes from (as the metadata for from A to B can be different than from A to C or from D to B so neither A alone nor B alone is enough to identify the metadata AFAICT). And if we imagine that the metadata is defined in attributes in A (MapTo) and B (MapFrom), things might become confusing if both classes have those attributes (which one wins, or how do we combine them ?)

Maybe we should only look for metadata in to target class to determine how to map the data? We want to create the target so let it determine how to interpret the source.

@joelwurtz
Copy link
Contributor

joelwurtz commented Apr 8, 2024

About metadata source of proof here is what we do in our actual library :

  • We process first metadata from source, as we consider it to be the main reference for Metadata
  • We also process metadata from target if available, but in case there is a conflict with metadata from source we throw an exception

A conflict happens when there is 2 ways of writing data to the same target field, we don't care about reading since we can read the same source fields for différent target fields, conflicts only happens when we want to write to the same field as we don't know which way we should use.

i.e. :

class Source {
   #[MapTo(key: 'value')]
   public $foo;
}

class Target {
   #[MapFrom(key: 'othervalue')]
   public $foo;
}

$mapper->map($source, Target::class); // throw Exception because there is 2 different definitions

From what i see this is only the path that can leads to conflict otherwise merging metadata from both source and target is not a problem

Also this problems only occurs we you want to use attribute, when using a different way of configuring metadata (like a php / yaml / xml file) it could be directly attached to a "from A to B" mapper so there is no way of having conlicts there.

Problems may comes if we want to allow "nullable" target or source in attribute metadata (like i want to use this configuration for all target / source). And in the same times we want a specific configuration for a specific mapping :

class Source {
   #[MapTo(key: 'value')]
   public $foo;
}

class Target {
   #[MapFrom(source: Source::class, key: 'othervalue')]
   public $foo;
}

In this case there is 3 approach :

  • we say that a specific metadata always overseed "global" one
  • we don't allow "global" mapping configuration and requires that an user always specify a target or a source in configuration
  • we implement a priority mecanism:
class Source {
   #[MapTo(key: 'value')] // Priority will default to 0 so not necessary here
   public $foo;
}

class Target {
   #[MapFrom(key: 'othervalue', priority: 10)]
   public $foo;
}

$mapper->map($source, Target::class); // don't throw an exception and use the Target configuration

With the priority mecanism we could still throw an exception if we have the same priority, but at least in this case it's something that the user can fix

@maxbeckers
Copy link
Contributor

At first, @joelwurtz you and your team did a great job and implemented a cool mapping tool.
Also the approach in #51741 is IMHO a great approach into the same direction.

Just what I think about:
These two approaches are dealing with attributes on the properties of the objects we want to map as we also know it from doctrine annotations / attributes in entities. So it's familiar to the symfony community and developers.
But I also like / prefer the approach of Mapstruct to implement separate interfaces (of in some php cases maybe also abstract classes) for separation of concerns by keeping mapping logic separate from domain objects. With this approach it's easier to realize different mappings to an object to move the data betweed different application layers. But yes, it would be a bit more boilerplate code and a new way of data handling in the symfony universe.

In the end I'm fine with both approaches, I just worked a lot in the last years with Java / Spring Boot and Mapstruct and like the approach of defining separated mappers even if also spring boot uses annotations e.g. on database entities.

@rogeriolino
Copy link

approach of Mapstruct

+1 for Mapstruct like approach, I had also commented about it here #51741 (comment)

Maybe adding a supports method to ObjectMapperInterface could help on it:

/**
 * @template T
 * @template K
 */
interface ObjectMapperInterface 
{
    /**
     * @param array|\stdClass|T $source
     * @param class-string<K>|'array'|K $target
     */
    public function supports(array|object $source, string|object $target): bool;

    /**
     * @param array|\stdClass|T $source
     * @param class-string<K>|'array'|K $target
     * @return array|K
     */
    public function map(array|object $source, string|object $target, array $context = []): array|object;
}

Bi-directional Mapper definition:

#[AsMapper]
interface CarMapperInterface
{
    #[Mapping(source: "numberOfSeats", target: "seatCount")]
    public function fromEntityToDto(Car $car): CarDto;

    #[Mapping(source: "seatCount", target: "numberOfSeats")]
    public function fromDtoToEntity(CarDto $car): Car;
}

// or

#[AsMapper]
abstract class CarMapper
{
    #[Mapping(source: "numberOfSeats", target: "seatCount")]
    public abstract function fromEntityToDto(Car $car): CarDto;

    #[Mapping(source: "seatCount", target: "numberOfSeats")]
    public abstract function fromDtoToEntity(CarDto $car): Car;

    // After mapping hooks can be defined when using abstract class 
    #[AfterMapping]
    public function afterDtoMapping(CarDto $car): void
    {
        // do anything
    }
}

Generated code

/** auto-generated code */
final class CarMapperImpl1 implements ObjectMapperInterface
{
    public function supports(array|object $source, string|object $target): bool
    {
        return get_class($source) === Car::class && get_class($target) === CarDto::class;
    }

    public function map(array|object $source, string|object $target, array $context = []): array|object
    {
        if (!$target instanceof CarDto::class) {
            $target = new CarDto();
        }
        $target->setMake($source->getMake());
        $target->setSeatCount($source->getNumberOfSeats());
        $target->setType($source->getType());

        return $target;
    }
}

/** auto-generated code */
final class CarMapperImpl2 implements ObjectMapperInterface
{
    public function supports(array|object $source, string|object $target): bool
    {
        return get_class($source) === CarDto::class && get_class($target) === Car::class;
    }

    public function map(array|object $source, string|object $target, array $context = []): array|object
    {
        if (!$target instanceof Car::class) {
            $target = new Car();
        }
        $target->setMake($source->getMake());
        $target->setNumberOfSeats($source->getSeatCount());
        $target->setType($source->getType());

        return $target;
    }
}

Mapper entrypoint:

// should it implement ObjectMapperInterface ???
class ObjectMapper implements ObjectMapperInterface
{
    public function __construct(
        private readonly iterable $mappers,
    ) {
    }

    public function supports(array|object $source, string|object $target): bool
    {
        // returning false to ignore itself on mappers loop (???)
        return false;
    }

    public function map(array|object $source, string|object $target, array $context = []): array|object
    {
        foreach ($this->mappers as $mapper) {
            if ($mapper->supports($source, $target) {
                return $mapper->map($source, $target);
            }
        }
        // mapping fallback (???)
    }
}

I feel the same way as @maxbeckers; I've spent a lot of time working with Spring Boot + Mapstruct, which is what makes us love it.

@soyuka
Copy link
Contributor

soyuka commented Apr 30, 2024

These two approaches are dealing with attributes on the properties of the objects we want to map as we also know it from doctrine annotations / attributes in entities. So it's familiar to the symfony community and developers.

Indeed this is why I feel that adding attributes on a class is quite good from a developer experience point of view. Still, I think that it's possible to have another approach even though it may not be as popular when you're working on RAD. As many mentioned in my PR any implementation needs a way to allow custom mapping so that it's possible to have different ways on how to declare what maps to what.

I'm against the supports pattern as in practice it creates way more loops then we actually need, especially when generating code. In other words you should be able to bind the mapper without having to loop over many mappers.

From our discussion around the Mapper subject, we agreed that at some point we'd have multiple ways of mapping things behind a single interface, I don't see why it couldn't be the same with how mapping is defined in the first place. I'll attempt a Map Struct approach on top of my PR to see if it fits with the Java MapStruct specification.

@maxbeckers
Copy link
Contributor

Hi @soyuka,

i don't mean to write any services with a supports method.

Let me try a short simple example.

class A {
  public $propertyA;
  public $propertyB;
  public $propertyC
}

class B {
  public $propertyC;
  public $propertyD;
}

interface AToBMapper {
  #[Mapping(source = 'propertyA', target = 'propertyD')]
  #[Mapping(source = 'propertyB', ignore = true)]
  public function mapAToB(A $a): B;
}

And this would then autogenerate a service AToBMapperImpl that is available in the Container. So you can add the Interface to your constructor and the service is autowired.

Service would look like this.

class AToBMapperImpl {
  public function mapAToB (A $a): B
  {
    $b = new B();
    $b->propertyC = $a->propertyC;
    $b->propertyD = $a->propertyA;
}

And of course you could also add a instance of B to map the data to an existing instance. Was this clear?

@soyuka
Copy link
Contributor

soyuka commented Apr 30, 2024

I understood @maxbeckers I was answering to @rogeriolino regarding the suggested implementation.

@rogeriolino
Copy link

yup, I've just suggested that supports because of this RFC post having an interface as entrypoint. Seems like it's defining that we would not inject the specific mapper but a generic one.

From my previous post, a CarService can inject CarMapperInterface $mapper directly without needing to deal with generic ObjectMapperInterface $mapper

@joelwurtz
Copy link
Contributor

From my point of view, you should always use the generic mapper interface, having a specific interface may be nice but will not work with library that don't know in advance which object comes from (i.e. api platform).

However i agree that attributes may be difficult to understand (since we are adding them on one or both classes)

But attributes are juste a way to configure metadata, i would prefer to follow the main symfony principle here : having the possibility to configure those metadata with php, xml or yaml config file

As an example for PHP it could be like this:

return static function (MapperConfigurator $configurator) {
    $configurator
        ->mapper(A::class, B::class)
             ->property('propertyA')->target('propertyD')->end()
             ->property('propertyB')->ignore()->end()
        ->end()
        ->mapper(B::class, A::class)
             ->property('propertyD')->target('propertyA')->end()
        ->end()
};

I think this approach is more consistent with the way that symfony handle things

@soyuka
Copy link
Contributor

soyuka commented May 2, 2024

I think that the main reason to be able to configure the mapper not using the source nor the target as a way to configure is when you don't own the object (ie it is from the vendors). As @joelwurtz said the metadata layer should be there to help configuring differently, not only using attributes. Still, in my implementation at #51741, the Map attribute is the DTO used to instruct the mapper. I've successfully implemented a MapStruct-like implementation on top of #51741, I don't know if we should provide both configuration approaches though.

@joelwurtz
Copy link
Contributor

I don't know if we should provide both configuration approaches though.

I think it's a possibility, however not on the current PR, we should first focus on a symfony-like approach. Then we could bring a new interface in another PR that would leverage the MapStruct approach.

I fear that having too many ways to do that will only bring more friction, IMO would be better to add different features step by step

@tsantos84
Copy link
Contributor

My contribution about this subject is that generated code always is faster than reflection at runtime. We need to consider this when look forward about this object mappers/serializers.

@carsonbot
Copy link

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?
Every feature is developed by the community.
Perhaps someone would like to try?
You can read how to contribute to get started.

@Korbeil
Copy link
Contributor Author

Korbeil commented Dec 5, 2024

This is not a stalled issue and is waiting on ObjectMapper being merged to move forward.

@carsonbot carsonbot removed the Stalled label Dec 5, 2024
@GromNaN
Copy link
Member

GromNaN commented Mar 25, 2025

PR merged #51741

Thanks @Korbeil, @soyuka, @joelwurtz and everyone who contributed to this new component.

@GromNaN GromNaN closed this as completed Mar 25, 2025
@GromNaN GromNaN reopened this Mar 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
RFC RFC = Request For Comments (proposals about features that you want to be discussed)
Projects
None yet
Development

No branches or pull requests