Skip to content

Confine SqlCursor to the critical section accessing the connection. #2132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 19 commits into from

Conversation

andersio
Copy link
Contributor

@andersio andersio commented Dec 30, 2020

This is a proposed solution to fix the read-your-writes consistency violation as described in the issue:

It is API breaking however, since it changes all method signatures that previously return a SqlCursor.

TL;DR

  • SqlCursor is no longer returned to the caller.

  • Instead, the caller must supply a (SqlCursor) -> R lambda that uses the cursor, which the generic lambda result R will be passed back to the caller.

  • The above API changes allow the driver to close the SqlCursor unconditionally, after invoking the supplied lambda. Most importantly, it can do so before the critical section on the connection ends.

    This guarantees that all active select statements are always closed, before the connection is reused by another caller/thread. Only when there is no active statement, the connection can then move its end mark forward for future reads to see newly written data.

Note that these changes are applicable to SQLite on other platforms as well.

Note

This is API breaking in both SqlDriver and Query<T>. Especially the latter, which exposes to library users a execute() returning a SqlCursor that can be closed asynchronously (or never).

Though it seems to me that executeAs*() are the ones intended to be used, not execute(). So the proposal here is to replace fun execute(): SqlCursor without deprecation.

@andersio andersio changed the title Confine SqlCursor to the critical section accessing the connection. (API breaking) Confine SqlCursor to the critical section accessing the connection. (API breaking proposal) Dec 30, 2020
@andersio andersio force-pushed the anders/fix-escaped-cursor branch from bce1d15 to f4f0518 Compare January 4, 2021 12:14
@kpgalligan
Copy link
Collaborator

Conceptually I like this. The execute is similar in concept as it just takes the bindings as a lambda and returns nothing. It would make the driver internal management somewhat simpler.

However, as an alternative to solve #2123, if we wanted to keep the API we could keep the db connection aligned to the thread until everything is complete. That would mean the user needs to be careful to properly close the cursor, but you could argue that's true now.

Will comment on the "close cursor" in the issue.

@AlecKazakova
Copy link
Collaborator

on first blush i like it. it will be a bit (likely on the order of a couple months) before we're ready to do an API-breaking release of sqldelight so at that time we can get this merged in.

@AlecKazakova AlecKazakova added this to the API Breaking milestone Feb 21, 2021
@andersio
Copy link
Contributor Author

Cool, I will proceed to update the tests. 👍

@andersio andersio force-pushed the anders/fix-escaped-cursor branch from 2850495 to b0848b2 Compare April 3, 2021 19:45
@andersio andersio marked this pull request as ready for review April 3, 2021 19:46
@andersio
Copy link
Contributor Author

andersio commented Apr 3, 2021

Some changes made:

  • Moved the mapper lambda to be ordered before parameters and binder, so that changes to the test fixtures are minimized.

  • Cleanups should happen unconditionally, no matter if the mapper lambda throws an exception or not. 8794acc, affecting only native, jdbc and js drivers.

@andersio andersio changed the title Confine SqlCursor to the critical section accessing the connection. (API breaking proposal) Confine SqlCursor to the critical section accessing the connection. Apr 3, 2021
@andersio andersio force-pushed the anders/fix-escaped-cursor branch from 0785e86 to 96c453e Compare April 3, 2021 23:32
@andersio andersio force-pushed the anders/fix-escaped-cursor branch 2 times, most recently from 5086701 to a27dd34 Compare April 7, 2021 10:05
@andersio andersio force-pushed the anders/fix-escaped-cursor branch from a27dd34 to 7cab47d Compare April 7, 2021 10:30
@andersio andersio force-pushed the anders/fix-escaped-cursor branch from fff878c to 2b50d14 Compare April 7, 2021 12:15
@andersio andersio force-pushed the anders/fix-escaped-cursor branch from 7800a77 to fe8b74a Compare April 7, 2021 13:52
@AlecKazakova
Copy link
Collaborator

We're now in API breaking territory, I re-reviewed the API changes and they still look good to me, are you able to rebase this one? If not I can give it a go at some point

@eygraber
Copy link
Contributor

@AlecStrong mind if I take a crack at rebasing?

@AlecKazakova
Copy link
Collaborator

👍 all yours, package name is probably the most painful one

@eygraber
Copy link
Contributor

I don't think I can push here, so I pushed to https://github.com/eygraber/sqldelight/tree/fix-escaped-cursor

@AlecKazakova
Copy link
Collaborator

ya just open a new pr so we can get ci going

@eygraber eygraber mentioned this pull request Apr 12, 2022
@AlecKazakova
Copy link
Collaborator

Merged here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

4 participants