Skip to content

Multi-context execution (py.Context) #144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 81 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
fc66b7f
py.CompileMode now a string enum
Dec 8, 2021
d10c35f
addressed benign Go warnings
Dec 8, 2021
2161b71
make Float implement M__index__
Dec 8, 2021
247ec42
added string helper functions
Dec 8, 2021
9a75271
updated to 1.15
Dec 8, 2021
90a8988
new Ctx-aware module model
Dec 9, 2021
c51644e
py.Ctx model allows concurrent execution
Dec 9, 2021
7f7d213
repl.New() now accepts an existing py.Ctx
Dec 9, 2021
708bb7e
fixed typo in callInternal()
Dec 9, 2021
a244256
unbroke test compatibilty
Dec 9, 2021
75d8743
cleaned up py.Ctx initialization wrt sys and DefaultCtxOpts()
Dec 10, 2021
d5fb8e8
added multi Ctx benchmark
Dec 10, 2021
269bdc0
WIP
Dec 10, 2021
4db9c65
fixed py.NewCtx not being set
Dec 10, 2021
53d05db
fixed List.sort() detection of instance invocation
Dec 10, 2021
7fd1c29
separated path resolution from path evaluation
Dec 11, 2021
8f3a104
removed cruft
Dec 18, 2021
5ebaf66
place ctx at top
Dec 18, 2021
d423190
err msg tweak
drew-512 Dec 18, 2021
b33c266
typo fix
drew-512 Dec 18, 2021
efca761
for loops we can all agree on
drew-512 Dec 18, 2021
1569475
removed outdated comments
Dec 18, 2021
a9c5f1f
Merge branch 'master' of https://github.com/drew-512/gpython
Dec 18, 2021
6372aee
added info for Frame.Ctx
Dec 19, 2021
4af4f90
removed unnecessary assignment
Dec 19, 2021
aec59f8
copyright and formatting
Dec 19, 2021
74f1a19
Ctx and NewCtx docs
Dec 19, 2021
8fa6622
switch cleanup
Dec 19, 2021
26fb9aa
leaner ModuleImpl schema
Dec 20, 2021
8614aeb
added GetDict() impl
Dec 20, 2021
86531c6
Compile() now returns py.Code (vs py.Object)
Dec 20, 2021
f71b961
tighter py.Ctx invocation patterns
Dec 20, 2021
77a7954
WIP
Dec 20, 2021
6bba7da
reverted to 644
Dec 20, 2021
46d1052
chmod 644
Dec 20, 2021
66fad23
enhanced RunFile() to enclose RunInNewModule()
Dec 27, 2021
95afd7b
Update examples/multi-ctx/main.go
drew-512 Jan 10, 2022
319edb7
Update modules/runtime.go
drew-512 Jan 10, 2022
06a2eae
Update modules/runtime.go
drew-512 Jan 10, 2022
8d4d586
Update modules/runtime.go
drew-512 Jan 10, 2022
d475e67
Update vm/eval.go
drew-512 Jan 10, 2022
6afac99
Update py/run.go
drew-512 Jan 10, 2022
931c346
Update modules/runtime.go
drew-512 Jan 10, 2022
a7ce4bb
Update modules/runtime.go
drew-512 Jan 10, 2022
c1f8809
Update modules/runtime.go
drew-512 Jan 10, 2022
9f02cf1
Update modules/runtime.go
drew-512 Jan 10, 2022
85e3a4a
Update modules/runtime.go
drew-512 Jan 10, 2022
fbec34e
Update modules/runtime.go
drew-512 Jan 10, 2022
dbd54d8
code cleanuo
Jan 10, 2022
71a5604
Module doc str
Jan 10, 2022
634823c
consistent arg order of NewFunction
Jan 10, 2022
6a0c11e
license and formatting
Jan 10, 2022
6242836
comment and cruft cleanup
Jan 10, 2022
61c7830
renamed Store to ModuleStore + docs
Jan 10, 2022
bb98aa8
added import docs
Jan 10, 2022
36b1018
shadowing cleanup
Jan 10, 2022
f2602bc
docs tweaks
Jan 10, 2022
e25e3bb
code cleanup
Jan 10, 2022
9421b6b
synced with gpython
Jan 10, 2022
92aed8a
Merge branch 'go-python:master' into master
drew-512 Jan 10, 2022
96515ea
Update modules/runtime.go
drew-512 Jan 10, 2022
f62f2b7
Update modules/runtime.go
drew-512 Jan 10, 2022
3bdba62
reverted from %w to %v (linux CI compile failure)
Jan 10, 2022
8033615
renamed py.Ctx to py.Context
Jan 10, 2022
681abf2
added Context.Close() and ModuleImpl.OnContextClosed()
Jan 18, 2022
7508556
docs and LoadIntsFromList
Jan 19, 2022
99bcb40
rename edits
Jan 21, 2022
0cc7650
push/popBusy now private
Jan 21, 2022
9afb2d3
doc edits
Jan 21, 2022
7d2a8d9
fixed kwarg issues in ParseTupleAndKeywords() and builtin_print test
Jan 25, 2022
ba7db80
added Flush kwarg to print test
Jan 25, 2022
61e42c8
added embedding example
Jan 26, 2022
168c337
main README makeover
Jan 26, 2022
2b7e1ab
fixed type conversion
Jan 26, 2022
3821b4b
Update README.md
drew-512 Jan 28, 2022
170f0d2
Update README.md
drew-512 Jan 28, 2022
b68b9f6
Update py/util.go
drew-512 Jan 28, 2022
759023b
Update py/util.go
drew-512 Jan 28, 2022
0c77716
Update py/util.go
drew-512 Jan 28, 2022
505755d
LoadIntsFromList cleanup
Jan 28, 2022
02d6b3e
comment edits
Jan 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleaned up py.Ctx initialization wrt sys and DefaultCtxOpts()
  • Loading branch information
Drew O'Meara committed Dec 10, 2021
commit 75d874323cff1fb50f8a1b9c65759f7c07347b85
4 changes: 2 additions & 2 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ func main() {
flag.Parse()
args := flag.Args()

opts := py.DefaultCtxOpts
opts.Args = flag.Args()
opts := py.DefaultCtxOpts()
opts.SysArgs = flag.Args()
ctx := py.NewCtx(opts)

if *cpuprofile != "" {
Expand Down
7 changes: 4 additions & 3 deletions modules/runtime.go
Original file line number Diff line number Diff line change
Expand Up @@ -137,9 +137,10 @@ func NewCtx(opts py.CtxOpts) py.Ctx {
ctx.store = py.NewStore()

py.Import(ctx, "builtins", "sys")
if opts.SetSysArgs {
ctx.Store().MustGetModule("sys").Globals["argv"] = py.NewListFromStrings(opts.Args)
}

sys_mod := ctx.Store().MustGetModule("sys")
sys_mod.Globals["argv"] = py.NewListFromStrings(opts.SysArgs)
sys_mod.Globals["path"] = py.NewListFromStrings(opts.SysPaths)

return ctx
}
Expand Down
41 changes: 30 additions & 11 deletions py/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,10 @@ const (
SmartCodeAcquire RunFlags = 0x01
)

// type Sys interface {
// ResetArfs
// }

// Rename to Ctx?
// Ctx is gpython virtual environment instance ("context"), providing a mechanism
// for multple gpython interpreters to run concurrently without restriction.
//
// See BenchmarkCtx() in vm/vm_test.go
type Ctx interface {

// These each initiate blocking execution.
Expand All @@ -28,12 +27,14 @@ type Ctx interface {
// Runs a given file in the given host module.
RunFile(runPath string, opts RunOpts) (*Module, error)

// // Execution of any of the above will stop when the next opcode runs
// Execution of any of the above will stop when the next opcode runs
// @@TODO
// SignalHalt()

//Sys() Sys

// Returns the named module for this context (or an error if not found)
GetModule(moduleName string) (*Module, error)

// WIP
Store() *Store
}

Expand All @@ -60,15 +61,33 @@ type RunOpts struct {
UseSysPaths bool
}

var DefaultCtxOpts = CtxOpts{}
// Can be changed during runtime and will \play nice with others using DefaultCtxOpts()
var CoreSysPaths = []string{
".",
"lib",
}

// Can be changed during runtime and will \play nice with others using DefaultCtxOpts()
var AuxSysPaths = []string{
"/usr/lib/python3.4",
"/usr/local/lib/python3.4/dist-packages",
"/usr/lib/python3/dist-packages",
}

type CtxOpts struct {
Args []string
SetSysArgs bool
SysArgs []string // sys.argv initializer
SysPaths []string // sys.path initializer
}

// High-level entry points
var (
DefaultCtxOpts = func() CtxOpts {
opts := CtxOpts{
SysPaths: CoreSysPaths,
}
opts.SysPaths = append(opts.SysPaths, AuxSysPaths...)
return opts
}
NewCtx func(opts CtxOpts) Ctx
Compile func(str, filename string, mode CompileMode, flags int, dont_inherit bool) (Object, error)
)
9 changes: 6 additions & 3 deletions pytest/pytest.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import (
"github.com/go-python/gpython/py"
)

var gCtx = py.NewCtx(py.DefaultCtxOpts)
var gCtx = py.NewCtx(py.DefaultCtxOpts())

// Compile the program in the file prog to code in the module that is returned
func compileProgram(t testing.TB, prog string) (*py.Module, *py.Code) {
Expand All @@ -35,14 +35,17 @@ func compileProgram(t testing.TB, prog string) (*py.Module, *py.Code) {
if err != nil {
t.Fatalf("%s: ReadAll failed: %v", prog, err)
}
return CompileSrc(t, gCtx, string(str), prog)
}

obj, err := compile.Compile(string(str), prog, py.ExecMode, 0, true)
func CompileSrc(t testing.TB, ctx py.Ctx, pySrc string, prog string) (*py.Module, *py.Code) {
obj, err := compile.Compile(string(pySrc), prog, py.ExecMode, 0, true)
if err != nil {
t.Fatalf("%s: Compile failed: %v", prog, err)
}

code := obj.(*py.Code)
module := gCtx.Store().NewModule(gCtx, py.ModuleInfo{
module := ctx.Store().NewModule(ctx, py.ModuleInfo{
FileDesc: prog,
}, nil, nil)
return module, code
Expand Down
2 changes: 1 addition & 1 deletion repl/repl.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ type UI interface {
// New create a new REPL and initialises the state machine
func New(ctx py.Ctx) *REPL {
if ctx == nil {
ctx = py.NewCtx(py.DefaultCtxOpts)
ctx = py.NewCtx(py.DefaultCtxOpts())
}

r := &REPL{
Expand Down
31 changes: 11 additions & 20 deletions sys/sys.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,15 +23,6 @@ import (
"github.com/go-python/gpython/py"
)

// Alter this list before the sys module starts to alter the factory path set
var SysPathInitializer = []string{
"./", // Denotes the dir of the current file (or cwd if __file__ is not set)
"./lib",
"/usr/lib/python3.4",
"/usr/local/lib/python3.4/dist-packages",
"/usr/lib/python3/dist-packages",
}

const module_doc = `This module provides access to some objects used or maintained by the
interpreter and to functions that interact strongly with the interpreter.

Expand Down Expand Up @@ -666,14 +657,22 @@ func init() {
stdout := &py.File{File: os.Stdout, FileMode: py.FileWrite}
stderr := &py.File{File: os.Stderr, FileMode: py.FileWrite}

executable, err := os.Executable()
if err != nil {
panic(err)
}

globals := py.StringDict{
"path": py.NewList(),
"argv": py.NewListFromStrings(os.Args[1:]),
"stdin": stdin,
"stdout": stdout,
"stderr": stderr,
"__stdin__": stdin,
"__stdout__": stdout,
"__stderr__": stderr,
"executable": py.String(executable),

//"version": py.Int(MARSHAL_VERSION),
// /* stdin/stdout/stderr are now set by pythonrun.c */

Expand Down Expand Up @@ -798,21 +797,13 @@ func init() {
// #endif
}

executable, err := os.Executable()
if err != nil {
panic(err)
}

globals["executable"] = py.String(executable)
globals["path"] = py.NewListFromStrings(SysPathInitializer)

py.RegisterModule(&py.StaticModule{
Info: py.ModuleInfo{
Name: "sys",
Doc: module_doc,
Name: "sys",
Doc: module_doc,
},
Methods: methods,
Globals: globals,
})

}