Skip to content

synchronous calls in goroutine blocks other concurrent goroutines #849

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pjebs opened this issue Aug 10, 2018 · 16 comments
Closed

synchronous calls in goroutine blocks other concurrent goroutines #849

pjebs opened this issue Aug 10, 2018 · 16 comments

Comments

@pjebs
Copy link
Contributor

pjebs commented Aug 10, 2018

I have 1 goroutine which contains a call to a synchronous function.

I have many other independent concurrent goroutines which ordinarily would run concurrently. But they don't run until the synchronous function returns.

@pjebs
Copy link
Contributor Author

pjebs commented Aug 10, 2018

I also have a:

		js.Global.Call("setInterval", func() {
			println("test 1")
		}, 1000)

This is outside of any goroutines I have created. (assume it is called from main).
This continually works and is not blocked. This implies the main runloop itself is not blocked.

@theclapp
Copy link

theclapp commented Aug 11, 2018

I have 1 goroutine which contains a call to a synchronous function.

What does the synchronous function look like / do? Is it Go or regular Javascript?

js.Global.Call("setInterval", func() {
	println("test 1")
}, 1000)

This continually works and is not blocked. This implies the main runloop itself is not blocked.

I'm not sure I agree. This would run as regular Javascript outside the Go runloop. If you had Go code that said

go func() {
    for {
        println("test 1")
        time.Sleep(time.Second)
    }
}()

and that kept running, I think that would imply that the Go runloop is not blocked.

@pjebs
Copy link
Contributor Author

pjebs commented Aug 11, 2018

The synchronous function is a non-Go function (electron function).

@pjebs
Copy link
Contributor Author

pjebs commented Aug 11, 2018

Do you have any ideas why a synchronous function inside a goroutine would prevent other independent goroutines from running? That's definitely not the case in standard go.

@pjebs
Copy link
Contributor Author

pjebs commented Aug 11, 2018

I can also see that the synchronous function is effecting the network request from returning:

I have a different goroutine with a polyfilled net.http.Client.Do( req ). The request doesn't return until the synchronous function returns: #839

@dmitshur

@theclapp
Copy link

theclapp commented Aug 15, 2018

Do you have any ideas why a synchronous function inside a goroutine would prevent other independent goroutines from running? That's definitely not the case in standard go.

The synchronous function is not Go, so it blocks the Go scheduler.

It's as if in regular Go (before, I wanna say, 1.5?) you set MAXPROCS to 1 and then started a goroutine that just sat in a loop doing no function calls. The scheduler doesn't run.

GopherJS is not Go (though obviously it does its best). It compiles to Javascript. Javascript is single threaded. If some Javascript function blocks the rest of the Javascript engine, then there's nothing magical about GopherJS that can get around that, it'll block too.

Edit: I re-read this issue after commenting and I'm not as sure as I was. I think it's going to be hard to diagnose here without showing example code. Can you boil it down to a minimal working example?

@pjebs
Copy link
Contributor Author

pjebs commented Aug 16, 2018

The non-synchronous function doesn't seem to block entire javascript engine:
electron/electron#13820 (comment)
In the non-go version of my code, the console.log('test'); still runs continually.

A redacted Go-version (which is what I actually use):


js.Global.Call("setInterval", func() {
	println("test") // This is not blocked by synchronous function
}, 1000) 


popupMenu := func() {
	// Show standard menu
	go func() {
		contextMenu := mm.buildMainMenu()
		// This blocks all goroutines until menu is closed
		mm.tray.PopUpContextMenu(contextMenu, &electron.TrayPopUpContextMenuPosition{})
	}()
}

mm.tray.On(electron.EvtTrayClick, func(args ...*js.Object) { go popupMenu() })

@theclapp
Copy link

theclapp commented Aug 16, 2018

setTimeout not working reliably may be relevant: GopherJS uses setTimeout() to schedule goroutines.

Instead of testing with setInterval, can you try a setTimeout background loop, and see if that pauses or runs erratically?

When you say the main thread is blocked, how long do you wait? electron/electron#7079 (comment) mentions 10+ second delays.

@pjebs
Copy link
Contributor Author

pjebs commented Aug 17, 2018

I added:

	js.Global.Call("setInterval", func() {
		println("test1") // This works
	}, 1000)

	go func() {
		for {
			js.Global.Call("setTimeout", func() {
				println("test2") // This is blocked until menu is closed
			}, 1000)
			time.Sleep(time.Second)
		}
	}()

I tried setTimeout but until the menu is closed, the timer stops running. It restarts again after menu is closed.

@theclapp
Copy link

I think to test if setTimeout is broken, the setTimeout'd function would have to reset itself. Something along the lines of (untested)

var f func()
f = func() {
    println("test2")
    js.Global.Call("setTimeout", f, 1000)
}
f()

As it is, you've got a stalled goroutine, which we already know doesn't work.

I was also thinking that, since setInterval (apparently?) does still work, you might also try

js.Global.Call("setInterval", func() {
	runtime.Gosched()
}, 1000)

on the theory that this might let the GopherJS scheduler run, but then I looked at the source to Gosched, and it just calls setTimeout (indirectly). Alas. That said, you might give it a shot, just to make sure it won't help.

@pjebs
Copy link
Contributor Author

pjebs commented Aug 18, 2018

Gosched just caused a panic stating cannot block in JavaScript callback, fix by wrapping code in goroutine

@pjebs
Copy link
Contributor Author

pjebs commented Aug 18, 2018

I still think this is a gopherjs bug that a call to a synchronous function is able to block all independent goroutines.

@pjebs
Copy link
Contributor Author

pjebs commented Aug 18, 2018

I made a discovery. I replaced the synchronous code with a blocking Go function and it works.

I replaced original code:

js.Global.Call("setInterval", func() {
	println("test") // This is not blocked by synchronous function
}, 1000) 


popupMenu := func() {
	// Show standard menu
	go func() {
		contextMenu := mm.buildMainMenu()
		// This blocks all goroutines until menu is closed
		mm.tray.PopUpContextMenu(contextMenu, &electron.TrayPopUpContextMenuPosition{})
	}()
}

mm.tray.On(electron.EvtTrayClick, func(args ...*js.Object) { go popupMenu() })

Now:


js.Global.Call("setInterval", func() {
	println("test") // This is not blocked by synchronous function
}, 1000) 

block := func() {
	time.Sleep(30 * time.Second)
}

popupMenu := func() {
	// Show standard menu
	go func() {
		contextMenu := mm.buildMainMenu()
		// This works - doesn't block other independent goroutines
		block()
	}()
}

mm.tray.On(electron.EvtTrayClick, func(args ...*js.Object) { go popupMenu() })

@pjebs
Copy link
Contributor Author

pjebs commented Aug 18, 2018

The block function "simulates" a synchronous function and it works well.

@dmitshur @theclapp It seems like gopher js Scheduler doesn't know how to deal with synchronous non-Go functions. Shouldn't this be considered a bug? Is it a simple fix? Where in the code should I start looking?

@pjebs
Copy link
Contributor Author

pjebs commented Aug 18, 2018

The non-Go function is an electron function which I suspect is a node.js function which wraps a Objective-C++ function.

@theclapp
Copy link

Thank you for verifying that Gosched wouldn't work. It broke for a different reason than I expected, but whatever. :)


The block function you tested in place of mm.tray.PopUpContextMenu is not actually a blocking call. It sleeps. In GopherJS, time.Sleep is implemented in terms of setTimeout (which we're theorizing is broken):

func Sleep(d Duration) {
	c := make(chan struct{})
	js.Global.Call("$setTimeout", js.InternalObject(func() { close(c) }), int(d/Millisecond))
	<-c
}

I've played around for a while, simulating a blocking function with

block := func() {
	for n := 0; n < busyloop; n++ {
		_ = n
	}
}

(where busyloop is a variable I can increase outside of the blocked function), and different ways of calling time.Sleep or runtime.Gosched or js.Global.Call("$runScheduled"), and can't ever get a setInterval'd function to "break in" in the middle of some other goroutine.

It does appear that you can call $runScheduled yourself, and I never got a panic or anything (though that might be a false negative), but the runtime doesn't really expect that, and it didn't help. Or at least I couldn't get it to make a difference. It might in your case, since you have a different type of blocking call, with other real goroutines going on that are doing real things (as opposed to mine that were mostly twiddling their metaphorical thumbs).

So here's what I tried if you want to try it:

js.Global.Call("setInterval",
	js.InternalObject(js.Global.Get("$runScheduled")),
	10)

which compiles to

$global.setInterval($runScheduled, 10);

@dmitshur I'm about out of ideas. Please take a look at this when you get a chance. I'd recommend you take special notice of electron/electron#7079, which discusses setTimeout being fairly broken in Electron, @pjebs's current platform.

@pjebs pjebs closed this as completed Dec 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants