Skip to content

port-javascript: can't push twice #4860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Closed

port-javascript: can't push twice #4860

wants to merge 11 commits into from

Conversation

pmp-p
Copy link
Contributor

@pmp-p pmp-p commented Jun 20, 2019

would raise syntax error when importing externals.

would raise syntax error when importing externals.
@pmp-p
Copy link
Contributor Author

pmp-p commented Jun 20, 2019

patch is invalid it's just to show what will fail when going from do_load_from_lexer() there

mp_parse_compile_execute(lex, MP_PARSE_FILE_INPUT, mod_globals, mod_globals);

@pmp-p
Copy link
Contributor Author

pmp-p commented Jun 20, 2019

that seems to work https://gist.github.com/pmp-p/d72c8dddc8c3e28072b64176fa6a2492 but could be only for one code path in a test case. I leave that to any emscripten and core expert around.

@pmp-p
Copy link
Contributor Author

pmp-p commented Jun 22, 2019

no idea if there's a connexion to the nlr_push(&nlr) return value, but since recent emsdk update i get (on WASM builds not WASM=0 like javascript port) Linking globals named 'nlr_push_tail': symbol multiply defined!
/edit: fixed linking that's not the cause, nlr_push/nlr_pop block is still weird but only on one codepath

@pmp-p pmp-p changed the title javascript: can't push twice port-javascript: can't push twice Jun 23, 2019
@dpgeorge
Copy link
Member

would raise syntax error when importing externals.

Can you please expand on this, and also provide a way to reproduce the error that you see?

@pmp-p
Copy link
Contributor Author

pmp-p commented Jun 24, 2019

@dpgeorge just drop and compile https://github.com/pmp-p/micropython-ports-wasm into ports folder and launch the python mini server provided with and goto to http://127.0.0.1:8000/next.html
then type "import asset", a one line module that contains print("*assets.py*") that got built in emscripten FS. edit/ not "import assets" , namespace packages are ok i did not have '' in my sys.path when i wrote first.

edit/ with all emsdk version i have i get some "syntax error" on the first non empty / non comment line.

edit/ problem is raised by the parser because a module with a zero sized file - or with comments does not raise syntax error.

I can't explain to myself why at all apart from emsdk problem since the imp workaround here https://github.com/pmp-p/micropython-ports-wasm/blob/master/assets/imp.py can build a good module via its source even with untouched micropython git source.

edit/ parts of imp that use exec are there https://github.com/pmp-p/micropython-ports-wasm/blob/087b7d674c6d65e0b0ea28011e161caa8e4609cf/main.c#L180. AFAIK exec uses the same mechanism as import so it's really puzzling.

@pmp-p
Copy link
Contributor Author

pmp-p commented Jun 24, 2019

i added empty.py and asset.py into assets folder,

namespace packages and empty modules or containing '#' are fine ( and still import assets.assets raise the syntax too error i checked )
so it's the parser, empty module object creation is ok.

MicroPython 4913cdb-dirty on 2019-06-24; emscripten with wasm
>>> import empty
>>> import assets
>>> import asset
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "assets/asset.py", line 2
SyntaxError: invalid syntax
>>> dir(empty)
['__class__', '__name__', '__file__']
>>> dir(assets)
['__class__', '__name__', '__path__']
>>> dir(asset)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name not defined
>>> 

@pmp-p
Copy link
Contributor Author

pmp-p commented Jul 5, 2019

i think it has to do with memory allocation in parser lexer.

#4723 is failing too but in a different way.

@pmp-p
Copy link
Contributor Author

pmp-p commented Jul 5, 2019

@dpgeorge i found something weird by changing mp_lexer_show_token to

printf("(" UINT_FMT ":" UINT_FMT ") kind:%u str:%s len:%lu vstr.len:%zu", (uint)lex->tok_line, (uint)lex->tok_column, (uint)lex->tok_kind, lex->vstr.buf,strlen(lex->vstr.buf), lex->vstr.len);

and calling it there https://github.com/micropython/micropython/blob/master/py/parse.c#L1142
i get a 0 sized vstr len though it points to a correct c string

MicroPython 6f611c2-dirty on 2019-07-05; emscripten with wasm
>>> import xpy
(2:4) kind:1 str:try len:3 vstr.len:0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "xpy/__init__.py", line 2
SyntaxError: invalid syntax
>>> 

does 0 means the token was invalid or that vstr pointer failed to advance ?

@dpgeorge
Copy link
Member

dpgeorge commented Jul 5, 2019

i get a 0 sized vstr len though it points to a correct c string

That's allowed. If the kind is MP_TOKEN_INVALID (=1) then vstr is not necessarily valid. Only certain tokens are associated with a valid vstr. The "try" value you see in the vstr is from the previous valid token.

@dpgeorge
Copy link
Member

dpgeorge commented Jul 5, 2019

#4723 is failing too but in a different way.

You might need to combine both that patch (explicit root stack) and NLR removal #4131

@pmp-p
Copy link
Contributor Author

pmp-p commented Jul 5, 2019

The "try" value you see in the vstr is from the previous valid token

@dpgeorge thx now i know what to search for, i guess lexer buffer got corrupted while parsing.
i hope it's somewhere in my file handling code.

@pmp-p
Copy link
Contributor Author

pmp-p commented Jul 5, 2019

pfffff i used fread instead of fgets lexer was running after source code len

well at least i'll keep that e615932

@pmp-p pmp-p closed this Jul 5, 2019
@pmp-p pmp-p deleted the patch-5 branch July 13, 2019 03:09
tannewt added a commit to tannewt/circuitpython that referenced this pull request Jun 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants