Skip to content

Multi-context execution (py.Context) #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 81 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
fc66b7f
py.CompileMode now a string enum
Dec 8, 2021
d10c35f
addressed benign Go warnings
Dec 8, 2021
2161b71
make Float implement M__index__
Dec 8, 2021
247ec42
added string helper functions
Dec 8, 2021
9a75271
updated to 1.15
Dec 8, 2021
90a8988
new Ctx-aware module model
Dec 9, 2021
c51644e
py.Ctx model allows concurrent execution
Dec 9, 2021
7f7d213
repl.New() now accepts an existing py.Ctx
Dec 9, 2021
708bb7e
fixed typo in callInternal()
Dec 9, 2021
a244256
unbroke test compatibilty
Dec 9, 2021
75d8743
cleaned up py.Ctx initialization wrt sys and DefaultCtxOpts()
Dec 10, 2021
d5fb8e8
added multi Ctx benchmark
Dec 10, 2021
269bdc0
WIP
Dec 10, 2021
4db9c65
fixed py.NewCtx not being set
Dec 10, 2021
53d05db
fixed List.sort() detection of instance invocation
Dec 10, 2021
7fd1c29
separated path resolution from path evaluation
Dec 11, 2021
8f3a104
removed cruft
Dec 18, 2021
5ebaf66
place ctx at top
Dec 18, 2021
d423190
err msg tweak
drew-512 Dec 18, 2021
b33c266
typo fix
drew-512 Dec 18, 2021
efca761
for loops we can all agree on
drew-512 Dec 18, 2021
1569475
removed outdated comments
Dec 18, 2021
a9c5f1f
Merge branch 'master' of https://github.com/drew-512/gpython
Dec 18, 2021
6372aee
added info for Frame.Ctx
Dec 19, 2021
4af4f90
removed unnecessary assignment
Dec 19, 2021
aec59f8
copyright and formatting
Dec 19, 2021
74f1a19
Ctx and NewCtx docs
Dec 19, 2021
8fa6622
switch cleanup
Dec 19, 2021
26fb9aa
leaner ModuleImpl schema
Dec 20, 2021
8614aeb
added GetDict() impl
Dec 20, 2021
86531c6
Compile() now returns py.Code (vs py.Object)
Dec 20, 2021
f71b961
tighter py.Ctx invocation patterns
Dec 20, 2021
77a7954
WIP
Dec 20, 2021
6bba7da
reverted to 644
Dec 20, 2021
46d1052
chmod 644
Dec 20, 2021
66fad23
enhanced RunFile() to enclose RunInNewModule()
Dec 27, 2021
95afd7b
Update examples/multi-ctx/main.go
drew-512 Jan 10, 2022
319edb7
Update modules/runtime.go
drew-512 Jan 10, 2022
06a2eae
Update modules/runtime.go
drew-512 Jan 10, 2022
8d4d586
Update modules/runtime.go
drew-512 Jan 10, 2022
d475e67
Update vm/eval.go
drew-512 Jan 10, 2022
6afac99
Update py/run.go
drew-512 Jan 10, 2022
931c346
Update modules/runtime.go
drew-512 Jan 10, 2022
a7ce4bb
Update modules/runtime.go
drew-512 Jan 10, 2022
c1f8809
Update modules/runtime.go
drew-512 Jan 10, 2022
9f02cf1
Update modules/runtime.go
drew-512 Jan 10, 2022
85e3a4a
Update modules/runtime.go
drew-512 Jan 10, 2022
fbec34e
Update modules/runtime.go
drew-512 Jan 10, 2022
dbd54d8
code cleanuo
Jan 10, 2022
71a5604
Module doc str
Jan 10, 2022
634823c
consistent arg order of NewFunction
Jan 10, 2022
6a0c11e
license and formatting
Jan 10, 2022
6242836
comment and cruft cleanup
Jan 10, 2022
61c7830
renamed Store to ModuleStore + docs
Jan 10, 2022
bb98aa8
added import docs
Jan 10, 2022
36b1018
shadowing cleanup
Jan 10, 2022
f2602bc
docs tweaks
Jan 10, 2022
e25e3bb
code cleanup
Jan 10, 2022
9421b6b
synced with gpython
Jan 10, 2022
92aed8a
Merge branch 'go-python:master' into master
drew-512 Jan 10, 2022
96515ea
Update modules/runtime.go
drew-512 Jan 10, 2022
f62f2b7
Update modules/runtime.go
drew-512 Jan 10, 2022
3bdba62
reverted from %w to %v (linux CI compile failure)
Jan 10, 2022
8033615
renamed py.Ctx to py.Context
Jan 10, 2022
681abf2
added Context.Close() and ModuleImpl.OnContextClosed()
Jan 18, 2022
7508556
docs and LoadIntsFromList
Jan 19, 2022
99bcb40
rename edits
Jan 21, 2022
0cc7650
push/popBusy now private
Jan 21, 2022
9afb2d3
doc edits
Jan 21, 2022
7d2a8d9
fixed kwarg issues in ParseTupleAndKeywords() and builtin_print test
Jan 25, 2022
ba7db80
added Flush kwarg to print test
Jan 25, 2022
61e42c8
added embedding example
Jan 26, 2022
168c337
main README makeover
Jan 26, 2022
2b7e1ab
fixed type conversion
Jan 26, 2022
3821b4b
Update README.md
drew-512 Jan 28, 2022
170f0d2
Update README.md
drew-512 Jan 28, 2022
b68b9f6
Update py/util.go
drew-512 Jan 28, 2022
759023b
Update py/util.go
drew-512 Jan 28, 2022
0c77716
Update py/util.go
drew-512 Jan 28, 2022
505755d
LoadIntsFromList cleanup
Jan 28, 2022
02d6b3e
comment edits
Jan 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
py.CompileMode now a string enum
  • Loading branch information
Drew O'Meara committed Dec 8, 2021
commit fc66b7f649de357b91ba69b15d09cdc4574e2b1d
3 changes: 1 addition & 2 deletions compile/compile.go
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ func init() {
// the effects of any future statements in effect in the code calling
// compile; if absent or zero these statements do influence the compilation,
// in addition to any features explicitly specified.
func Compile(str, filename, mode string, futureFlags int, dont_inherit bool) (py.Object, error) {
func Compile(str, filename string, mode py.CompileMode, futureFlags int, dont_inherit bool) (py.Object, error) {
// Parse Ast
Ast, err := parser.ParseString(str, mode)
if err != nil {
Expand Down Expand Up @@ -1342,7 +1342,6 @@ func (c *compiler) NameOp(name string, ctx ast.ExprContext) {
default:
panic("NameOp: ctx invalid for name variable")
}
break
}
if op == 0 {
panic("NameOp: Op not set")
Expand Down
2 changes: 1 addition & 1 deletion compile/compile_data_test.go
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (

var compileTestData = []struct {
in string
mode string // exec, eval or single
mode py.CompileMode
out *py.Code
exceptionType *py.Type
errString string
Expand Down
2 changes: 1 addition & 1 deletion parser/grammar_test.go
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ var debugLevel = flag.Int("debugLevel", 0, "Debug level 0-4")
func TestGrammar(t *testing.T) {
SetDebug(*debugLevel)
for _, test := range grammarTestData {
Ast, err := ParseString(test.in, test.mode)
Ast, err := ParseString(test.in, py.CompileMode(test.mode))
if err != nil {
if test.exceptionType == nil {
t.Errorf("%s: Got exception %v when not expecting one", test.in, err)
Expand Down
16 changes: 8 additions & 8 deletions parser/lexer.go
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -62,20 +62,20 @@ type yyLex struct {
// can be 'exec' if source consists of a sequence of statements,
// 'eval' if it consists of a single expression, or 'single' if it
// consists of a single interactive statement
func NewLex(r io.Reader, filename string, mode string) (*yyLex, error) {
func NewLex(r io.Reader, filename string, mode py.CompileMode) (*yyLex, error) {
x := &yyLex{
reader: bufio.NewReader(r),
filename: filename,
indentStack: []int{0},
state: readString,
}
switch mode {
case "exec":
case py.ExecMode:
x.queue(FILE_INPUT)
x.exec = true
case "eval":
case py.EvalMode:
x.queue(EVAL_INPUT)
case "single":
case py.SingleMode:
x.queue(SINGLE_INPUT)
x.interactive = true
default:
Expand Down Expand Up @@ -933,7 +933,7 @@ func SetDebug(level int) {
}

// Parse a file
func Parse(in io.Reader, filename string, mode string) (mod ast.Mod, err error) {
func Parse(in io.Reader, filename string, mode py.CompileMode) (mod ast.Mod, err error) {
lex, err := NewLex(in, filename, mode)
if err != nil {
return nil, err
Expand All @@ -952,12 +952,12 @@ func Parse(in io.Reader, filename string, mode string) (mod ast.Mod, err error)
}

// Parse a string
func ParseString(in string, mode string) (ast.Ast, error) {
func ParseString(in string, mode py.CompileMode) (ast.Ast, error) {
return Parse(bytes.NewBufferString(in), "<string>", mode)
}

// Lex a file only, returning a sequence of tokens
func Lex(in io.Reader, filename string, mode string) (lts LexTokens, err error) {
func Lex(in io.Reader, filename string, mode py.CompileMode) (lts LexTokens, err error) {
lex, err := NewLex(in, filename, mode)
if err != nil {
return nil, err
Expand All @@ -984,6 +984,6 @@ func Lex(in io.Reader, filename string, mode string) (lts LexTokens, err error)
}

// Lex a string
func LexString(in string, mode string) (lts LexTokens, err error) {
func LexString(in string, mode py.CompileMode) (lts LexTokens, err error) {
return Lex(bytes.NewBufferString(in), "<string>", mode)
}
2 changes: 1 addition & 1 deletion parser/lexer_test.go
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,7 @@ func TestLex(t *testing.T) {
for _, test := range []struct {
in string
errString string
mode string
mode py.CompileMode
lts LexTokens
}{
{"", "", "exec", LexTokens{
Expand Down
79 changes: 79 additions & 0 deletions py/run.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
package py

type CompileMode string

const (
ExecMode CompileMode = "exec"
EvalMode CompileMode = "eval"
SingleMode CompileMode = "single"
)

type RunFlags int32

const (
// RunOpts.FilePath is intelligently interepreted to load the appropriate pyc object (otherwise new code is generated from the implied .py file)
SmartCodeAcquire RunFlags = 0x01
)

// type Sys interface {
// ResetArfs
// }

// Rename to Ctx?
type Ctx interface {
// These each initiate blocking execution.
//RunAsModule(opts RunOpts) (*Module, error)
//RunCodeAsModule(code *Code, moduleName string, fileDesc string) (*Module, error)
Run(globals, locals StringDict, code *Code, closure Tuple) (res Object, err error)
RunFrame(frame *Frame) (res Object, err error)
EvalCodeEx(co *Code, globals, locals StringDict, args []Object, kws StringDict, defs []Object, kwdefs StringDict, closure Tuple) (retval Object, err error)

// // Execution of any of the above will stop when the next opcode runs
// SignalHalt()

//Sys() Sys

GetModule(moduleName string) (*Module, error)
Store() *Store
}

const (
SrcFileExt = ".py"
CodeFileExt = ".pyc"

DefaultModuleName = "__main__"
)

type StdLib int32

const (
Lib_sys StdLib = 1 << iota
Lib_time

CoreLibs = Lib_sys | Lib_time
)

type RunParams struct {
ModuleInfo ModuleInfo // Newly created module to execute within (if ModuleInfo.Name is nil, then "__main__" is used)
NoopOnFNF bool // If set and Pathname did not resolve, (nil, nil) is returned (vs an error)
Silent bool // If set and an error occurs, no error info is printed
Pathname string
}

var DefaultCtxOpts = CtxOpts{}

type CtxOpts struct {
Args []string
SetSysArgs bool
}

// Some well known objects
var (
NewCtx func(opts CtxOpts) Ctx

// // Called at least once before using gpython; multiple calls to it have no effect.
// // Called each time NewCtx is called
// Init func()
Compile func(str, filename string, mode CompileMode, flags int, dont_inherit bool) (Object, error)
Run func(ctx Ctx, params RunParams) (*Module, error)
)
2 changes: 1 addition & 1 deletion symtable/symtable_data_test.go
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (

var symtableTestData = []struct {
in string
mode string // exec, eval or single
mode py.CompileMode
out *SymTable
exceptionType *py.Type
errString string
Expand Down