Toward PyScript v5 #2111
Replies: 5 comments 4 replies
-
|
My specific requirements are heading down the PWA track. (Full application running in browser doing work locally-using workers for lengthy calcs).
|
Beta Was this translation helpful? Give feedback.
-
|
We're currently working on fully-offline PWAs in pyscript.com, so that should be available (as an opt-in) soon. |
Beta Was this translation helpful? Give feedback.
-
|
+1 on all your ideas @WebReflection! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for this @WebReflection ! Below a few questions/considerations.
Can you better elaborate? This proposal has a few layers/consequences that I'd like to make sure are very clear. There's the pure tech aspect (== remove polyscript, bring plugins layers (or something like that) to PyScript, etc...) but also the other "Product" side of it around what actual interpreters/PLs we want to support in the future, etc. Can you cover the different aspects of your proposal and what it concretely means?
Am I understanding the proposal correctly that this would mainly affect the top level API around the
+1. Would that be a different type of
Yes please. Back to my question above, how much of this overlaps with the previous item? |
Beta Was this translation helpful? Give feedback.
-
|
if we want to support other PLs we also need these PLs to have a close enough feature parity with Pyodide and MicroPython. Right now that's unfortunately not the case, so that branching also documentation, expectations and so on for other PLs so far not many had interest in using feels like a bit of a overhead we don't fully need. The current state is that Polyscript is 100% needed and used only for 2 out of 4 interpreters it's offering and those interpreters are anyway incapable of offering everything we do or need, including events on the DOM and whatnot. I am OK keeping things the way these are right now as polyscript has never been a real issue or bottleneck, but the fact I need to jump back and forward between two projects that are strictly coupled and don't benefit from all the extra things (Ruby or Lua in particular) and branches in other things (Pyodide and MicroPython) doesn't feel the most elegant way forward, it's instead branching a lot of logic (inevitably due different capabilities) and it might be easier to contribute or add features if only focused on PyScript needs. Again, this is the least of the problems and the least of the focus / interest I have ... all these ideas were for the long-term future, nothing is needed now.
correct. For PyScript, having that transparently available, specially on the Web where many JS modules or native APIs are already async, feels like the best default we can offer.
Dare I say, Ruby can only run async too, and so is for PHP ... if many interpreters landing on WASM are async only, we'd better just drop some sync related code and embrace that way, slightly simplifying Polyscript code too (or better, removing some code that would throw otherwise in other PLs). If we need, internally, to still access those sync APIs, we can always do that with ease.
it would be more like We asked Damien to provide some build that includes sqlite3 and Pyodide has that already but AFAIK (and IIRC) both are volatile, erasing somehow the beauty and usefulness of SQLite 3 as db. Arguably though, I already have persistent SQLite 3 as JS module, so that we could even lazily provide that and see how it goes, before re-implementing something more suitable, or easier to embed, or more Pythonic, for our stack. Same goes for the Persistent FS ... about DuckDB I think it's based on SQLite anyway but for NoSQL solutions a key/value store like |
Beta Was this translation helpful? Give feedback.
-
|
never mind, I finally got where the question around DuckDB came from ... well, here an example of how we could deliver it too: we write a nice to embed package and we host it in our CDN for PyScript users and we document it, example: https://pyscript.com/@agiammarchi/jolly-silence-copy/latest |
Beta Was this translation helpful? Give feedback.
-
|
Ok, so... In regards to the topics:
|
Beta Was this translation helpful? Give feedback.
-
|
If we had the "File System Access API", could we not host the sql db on user's host. Providing some persistence. |
Beta Was this translation helpful? Give feedback.
-
|
Update beside the fact top-level-await is done (since quite a while ago) I have investigated the sqlite3 case and ...
Accordingly with the current state of affairs, OPFS is entirely async, but the sqlite3 WASM port is everything but async. For the time being, I've implemented a solution that uses the same
Basically, going OPFS would benefit only one use case: the user doesn't have enough RAM to handle the whole DB in memory ... but because there is quota associated, I am not expecting such DB to be that huge or issues will raise anyway and, on top of that, not using the available RAM means nothing can be done synchronously. In few words, if we are OK exploring this field synchronously for either main or workers use cases, I believe I have exactly what we need already to land an sqlite3 db that works consistently between Pyodide and MicroPython too, but if we really want to pursue the OPFS way, the whole API is going to be inevitably async for every single operation, even if in workers such sync behavior could be abstracted away (Atomics) so that it doesn't have to be that way ... but that opens a variant of the same storage between main and workers, while the current implementation I have would work seamlessly well on both cases. |
Beta Was this translation helpful? Give feedback.
-
Background
I know we use CalVer for versioning but, behind the scene, PyScript uses semver convention to land on npm in its "canary" or "developer preview" fashion.
Maybe not everyone payed enough attention to this tiny detail but we are at version 0.4.53 right now.
I've tried to stick with semver logic to date so that the minor number incremented only on breaking changes ... which ultimately never happened in recent months, all patches are features, as that's how it should be behind a
v0project.Current State
Thanks to the team and everyone else involved within, or around, this project, we have reached some sort of maturity where most new things are either features or fixes and no drastic changes are usually required (until they are ... but we already cover tons of different use cases and we should be proud of that, imho).
We've also kept hearing, and learning from, the community that on every valuable occasion mentioned pain points and/or solutions to common cases.
This discussion is around all the learning that ourselves, or our users, gained over the last year(s) of development and I would like to propose the only sensible breaking (yet, not really) changes that could bump minor before it's too late (we should've learned from outdated docs, examples, social media press, articles that won't work anymore, and so on ...).
The Future
I would like to summarize in here what I think would be an even more wonderful future for this project, listing point after point where I think everyone would win in the long term and commenting on each point too.
Everyone feedback would be more than appreciated and nothing is written in stones here, it's just a couple of points that keep bugging me daily and that are, imho, worth considering, or discussing.
runPythonAsyncthat allows top-level await, yet we execute everything behindrunPythoninstead for no reason whatsoever. We provide a<script type="py/mpy">convention that easily simulate or replace<script type="module">in the standard Web world, yet we don't handle that entry point as something anyone could await on our exposedasyncAPIs, such as fetch, recently storage, and whatnot. We also have a lot of logic involving hooks, where we need to disambiguate withasyncthose that are meant to run in sync VS async code, yet they lead to repeated hooks for no real-world benefit to anyone. Here I am saying we should get rid of any sync VS async disambiguation and just always run async by default. I am not super sure we should offer a sync version of anything, as when you write code in your HTML nobody is waiting for that last line result, so we can just call it a day and drop all "brain-complexity" or disambiguation needed to write hooks. Hooks to date have been untouched by our users: too complex to reason about and not portable across projects, due the fact once a script hasasyncattribute, they might even never be executed. Last, but not least, there are users confused by theasyncattribute as if it could ever break theirsyncexpectations, so it's clear to me having alwaysasynccode executed would be a win.pyscript.storagebut for anything more advanced or heavy, that might not be the best we can do. I've recently learned the originalsqlite3WASM target project uses Opaque FileSystem API to never need the whole thing in memory (RAM). It can actually run out of FS related operations through that opaque FS so it would play better on both mobile and more constrained (RAM speaking) devices. I think these two are not mutually exclusive, but I also think we should not interfere much with native SQLite potentials: we provide its API as it is and we grant persistent results on users' sessions after reloads or any other operation. I don't have in mind the whole API right now but I think we should provide a module, or helper, that does the right thing for our users without being just virtual, like it is for the FileSystem we're using instead.pyscript.storagealready out, nothing really stops us from providing, through the same mechanism, a persistent FileSystem that will automatically inject on bootstrap those persistent files in the virtual FS, with no need to have these listed in aconfig. This would enable, to me, a closer 1:1 solution for anything that needs persistent data and yet prefer a FileSystem approach over any K/V based databases.Hopefully I've exposed my main "desires" for the future of the project, and I would be more than happy to add any other idea to this list (or remove some).
Beta Was this translation helpful? Give feedback.
All reactions