Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

best practice to load many relations ids from database #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vincentlejeune opened this issue Oct 29, 2018 · 3 comments
Closed

best practice to load many relations ids from database #34

vincentlejeune opened this issue Oct 29, 2018 · 3 comments

Comments

@vincentlejeune
Copy link

I try to implement a solution that uses graphql-java and dataloader over a database.

I would like to know the best pratice to load a many relations when the nested ids required a database query.

in the documentation example, imagine that we replace starWarsCharacter.getFriendsIds() by a database query getFriendIdsFromDB(starWarsCharacter).
The friendsDataFetcher would looks like :

 DataFetcher friendsDataFetcher = new DataFetcher() {
            @Override
            public Object get(DataFetchingEnvironment environment) {
                StarWarsCharacter starWarsCharacter = environment.getSource();
                List<String> friendIds = getFriendIdsFromDB(starWarsCharacter);
                return characterDataLoader.loadMany(friendIds);
            }
        };

In this case it generates again a N+1 fetch problem. For each Character we have to query for the friends ids.

How to efficiently manage that case with the dataloader?

@bbakerman
Copy link
Member

Really what you are doing here is getting the "friends ids" of the character AND then loading those friends. So we need a batch loader that does those 2 operations at the same time.

So I think you might want to combine them into one batch loader that takes a "characterID" and can find friends for them all in the one step

So imagine your batch loader is like this (psudeo java code)

BatchLoader bl  = characterIdsKeys -> {
   
     ResultSet rs = db.execute(' select from friends where friends.characterId in ::characterIdsKeys:: group by friends.characterId');

   // now run through the group set of friends per characterId and make Character objects from them
  return serviceToUnpackGroupedResultSet(rs)
}

Now it might be that your database tables can only "find the ids of friends" in one query and then find the friends themselves via another query. So be it. Use 2 queries. Use the SQL "in" operator to find results for multiple elements (eg the batch of characters). Use SQL groupBy or order by to make sure you can pack back the results into a list that matches the input keys

@vincentlejeune
Copy link
Author

Hi @bbakerman

Thanks for your reply. I create a dataloader for the relation between characterId and the list of Friend. It works like a charm !

Just to be sure to understand, to load friends by characterIds, i create a dataloader that store the list of friends by characterId (DataLoader<CharacterId, List<Friend>>). So, by doing this, i by pass a potential Friend's dataloader (DataLoader<String, Friend>).

Is it possible to load the friends id and next use a Friend's dataloader to get the Friends, all of that in an async way ?

@budjb
Copy link

budjb commented Nov 1, 2018

This issue actually looks like the same thing I'm asking about in #36. @bbakerman, do you have an example of how the batch loader you pseudo-coded above would be used? This differs from the examples, since in the examples the IDs of the objects you want to load are passed to the dataloader, whereas here it seems you need to pass the ID of the source object. I'm not sure how that should look in usage.

@bbakerman bbakerman reopened this Jul 31, 2021
@graphql-java graphql-java locked and limited conversation to collaborators Jul 31, 2021

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants