Skip to content

decorator with type hint breaks #4173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
lastmjs opened this issue Sep 19, 2022 · 12 comments
Open

decorator with type hint breaks #4173

lastmjs opened this issue Sep 19, 2022 · 12 comments
Assignees

Comments

@lastmjs
Copy link

lastmjs commented Sep 19, 2022

This code runs fine on my machine but not from within RustPython:

from typing import Any

def query(func: Any):
    return func

@query
def hello_world() -> int:
    return 1_000

When you remove the Any type hint the code executes fine in RustPython.

@lastmjs lastmjs changed the title decorator with Any type hint breaks decorator with type hint breaks Sep 19, 2022
@lastmjs
Copy link
Author

lastmjs commented Sep 19, 2022

This code is also breaking:

from typing import Any

def hello_world() -> Any:
    return 'Hello world!'

@lastmjs
Copy link
Author

lastmjs commented Sep 20, 2022

Here's the full code that breaks, this is a wasm32-unknown-unknown environment without wasm_bindgen:

use rustpython;

fn custom_getrandom(_buf: &mut [u8]) -> Result<(), getrandom::Error> {
    Ok(())
}

getrandom::register_custom_getrandom!(custom_getrandom);

#[ic_cdk_macros::query]
fn simple_query() -> String {
    let interpreter = rustpython::vm::Interpreter::without_stdlib(Default::default());

    let result = interpreter.enter(|vm| {
        let scope = vm.new_scope_with_builtins();

        vm.run_code_string(
            scope.clone(),
            r#"
from typing import Any

def hello_world() -> Any:
    return "Hello Dan and Ben!"

result = hello_world()
            "#,
            "".to_owned(),
        )
        .unwrap();

        let result_value = scope.globals.get_item("result", vm).unwrap();
        let result_string: String = result_value.try_into_value(vm).unwrap();

        result_string
    });

    result
}

@lastmjs
Copy link
Author

lastmjs commented Sep 20, 2022

Does this possibly have to do with rustpython::vm::Interpreter::without_stdlib? I am calling vm.new_scope_with_builtins() so I would think not. This is breaking on my local machine compiled natively and in wasm32-unknown-unknown running within an Internet Computer canister locally

@lastmjs
Copy link
Author

lastmjs commented Sep 20, 2022

I get an error like this:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: PyBaseException', query/src/main.rs:28:10

It would help if I could get some useful information out of PyBaseException, but I've tried in various ways without much success.

@youknowone
Copy link
Member

To make things clear, do you mean CPython when you said 'my machine'?

import typing requires stdlib because typing is a part of stdlib. I was trying to provide easier vm init with InterpreterConfig in #4147. The init code will look like: https://github.com/RustPython/RustPython/pull/4147/files#diff-b1a35a68f14e696205874893c07fd24fdb88882b47c23cc0e0c80a30c7d53759R257

Would this be ok?

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

On my machine, I mean compiling and running RustPython for my laptop computer, so probably x86. Usually I'm compiling to Wasm to run in an Internet Computer replica, which is a much stricter wasm32-unknown-unknown environment.

I will try your suggestions, I thought vm.scope_with_builtins or whatever it was called provided this std library

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

Hmmm...what if I were able to get a &mut to the vm from the interpreter? Would it then just persist its state? I don't think I need the enter mechanics since I'm in a Wasm environment. What do you think? I might try it on my fork at least.

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

I just need to do something like this?

    let interpreter = rustpython::vm::Interpreter::with_init(Default::default(), |vm| {
        vm.add_native_modules(rustpython::vm::stdlib::get_module_inits());
    });

I still get a runtime error but if I'm on the right track, this doesn't seem too difficult. Based on the API you have I would expect a rustpython::vm::Interpreter::with_stdlib.

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

I have turned all features off, do I need to enable a feature to get the stdlib to work?

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

I have in my Cargo.toml:

rustpython = { path = "../../../RustPython", default-features = false, features = ["stdlib"] }

Here's all of the code for a self-contained example:

use rustpython::{self, vm::convert::ToPyObject};

fn custom_getrandom(_buf: &mut [u8]) -> Result<(), getrandom::Error> {
    Ok(())
}

getrandom::register_custom_getrandom!(custom_getrandom);

#[ic_cdk_macros::query]
fn simple_query() -> String {
    // let interpreter = rustpython::vm::Interpreter::without_stdlib(Default::default());
    let interpreter = rustpython::vm::Interpreter::with_init(Default::default(), |vm| {
        vm.add_native_modules(rustpython::vm::stdlib::get_module_inits());
    });

    let result = interpreter.enter(|vm| {
        let scope = vm.new_scope_with_builtins();

        // vm.invoke(func, args)

        // let py_object_ref_string = "hello there".to_string().to_pyobject(vm);
        // let py_object_ref_bool = true.to_pyobject(vm);

        // vm.invoke(func, args)
        // scope = scope;

        // TODO importing and using Any breaks!
        vm.run_code_string(
            scope.clone(),
            r#"
from typing import Any

def hello_world() -> Any:
    return "Hello Dan and Ben!"

result = hello_world()
            "#,
            "".to_owned(),
        )
        .unwrap();

        let result_value = scope.globals.get_item("result", vm).unwrap();
        let result_string: String = result_value.try_into_value(vm).unwrap();

        result_string
    });

    result
}

Am I doing something wrong?

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

When running similar code using RustPython on my machine compiled to x86, I was able to figure out the error:

ModuleNotFoundError: No module named 'typing'
there was an error

Did I do something wrong with vm.add_native_modules(rustpython::vm::stdlib::get_module_inits());?

@lastmjs
Copy link
Author

lastmjs commented Sep 21, 2022

I have figured it out. This was very tricky to figure out. It might be good to add better documentation or refactor the code to make this easier. Also, this might be a bug, because I would expect stdlib to work in a wasm32-unknown-unknown environment.

So the problem seems to be that I needed to add the freeze-stdlib feature, along with the stdlib. I then needed to change my interpreter initialization to look like this:

let interpreter = rustpython::vm::Interpreter::with_init(Default::default(), |vm| {
        vm.add_frozen(rustpython_pylib::frozen_stdlib());
    });

So that worked when compiling and running the code on my machine, x86 Ubuntu Linux (I don't know what target that is). I haven't actually been able to get this running in my wasm32-unknown-unknown environment, because unfortunately adding in freeze-stdlib and the rustpython-pylib crate doubles my Wasm binary size. This is blocking me from deploying my canister to the Internet Computer, because binaries are limited to 2mb without gzip, and about 10mb with gzip. The binary before freeze-stdlib was ~7mb and now it's ~15mb, which is just too large.

So a couple questions:

  1. Is this a bug? Should stdlib just work with wasm32-unknown-unknown?
  2. Is there any way to make the final binary smaller when using freeze-stdlib?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants