Skip to content

Update multiprocess.py #6637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 9 commits into from
Closed

Conversation

Thomas00010111
Copy link

Removed dependency to gobject (could not install/import package)
added () to return call_back

@Thomas00010111
Copy link
Author

Updated in main repository.

@@ -79,6 +80,7 @@ def main():
pl.plot()
time.sleep(0.5)
raw_input('press Enter...')
# input('press Enter...') #Python3
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just make it conditional on sys.version_info?

Copy link
Author

@Thomas00010111 Thomas00010111 Jun 23, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also remove both lines, actually they do not add anything to this example.

@Thomas00010111
Copy link
Author

In my version, which I derived from this example, I am using set_xdata and set_ydata to update the plot. Replotting becomes very slow because of the growing self.x and self.y array, but I guess that would be a new example.

@@ -8,9 +8,8 @@
import numpy as np

import matplotlib
matplotlib.use('GtkAgg')
matplotlib.use('TkAgg')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you're using canvas API now, this might be portable enough to not require setting a specific backend at all.

@Thomas00010111
Copy link
Author

Wasn't sure if I needed the set a backend. Removed the line.

@jenshnielsen
Copy link
Member

@Thomas00010111 I tested it on OSX, It seems like not all the backends are fork safe on OSX

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().
Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

I suggest adding a comment about this along with a commented out backend selection. In my experience the qt backends work the best, i.e. add something like/

# not all backends may allow safe plotting from multiple threads 
# you can select a specific backend by uncommenting the line below
# and update the selected backend as needed
# matplotlib.use('QT5Agg')

where you removed the other backend selection

@Thomas00010111
Copy link
Author

Thanks for testing on OSX. In my case I need to use matplotlib.use('QT4Agg').

@jenshnielsen
Copy link
Member

Some more investigation reveals that the best solution to get this working reliably on OSX is to use a forkserver as the default process starter see https://pythonhosted.org/joblib/parallel.html#bad-interaction-of-multiprocessing-and-third-party-libraries for a similar example. Forkserver is however not available on Windows and is only available for python 3.4 and upwards so I suggest adding this as a comment something along the lines of:

if __name__ == '__main__':
    # The default way to start a process on OSX ('fork')
    # does not work well with many gui frameworks on OSX
    # if you use Python 3.4 or later you can uncomment the
    # two lines below to change the default start to
    # forkserver.
    # import multiprocessing as mp
    # mp.set_start_method('forkserver')
    main()

@Thomas00010111
Copy link
Author

Pasted your lines into the code. Uncommenting the lines gives an error message in my case. I am running Ubuntu 14.04 and Python 3.4. It seems that the forkserver is already the default.

Traceback (most recent call last):
File "/usr/lib/python3.4/multiprocessing/forkserver.py", line 184, in main
_serve_one(s, listener, alive_r, handler)
File "/usr/lib/python3.4/multiprocessing/forkserver.py", line 220, in _serve_one
code = spawn._main(child_r)
File "/usr/lib/python3.4/multiprocessing/spawn.py", line 115, in _main
prepare(preparation_data)
File "/usr/lib/python3.4/multiprocessing/spawn.py", line 226, in prepare
_fixup_main_from_path(data['init_main_from_path'])
File "/usr/lib/python3.4/multiprocessing/spawn.py", line 278, in _fixup_main_from_path
run_name="mp_main")
File "/usr/lib/python3.4/runpy.py", line 240, in run_path
pkg_name=pkg_name, script_name=fname)
File "/usr/lib/python3.4/runpy.py", line 96, in _run_module_code
mod_name, mod_spec, pkg_name, script_name)
File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/media/Boot/Users/MrFish/Source Code/Dropbox/Workspace_Linux/QuadcopterSimulator/multiprocessing_demo.py", line 9, in
mp.set_start_method('forkserver')
File "/usr/lib/python3.4/multiprocessing/context.py", line 231, in set_start_method
raise RuntimeError('context has already been set')
RuntimeError: context has already been set

@jenshnielsen
Copy link
Member

jenshnielsen commented Jun 27, 2016

The code setting the forkserver needs to be in the if __name__ == '__main__': section as in my example above. Otherwise you get that error

@Thomas00010111
Copy link
Author

Ah, ok. Changed the code and the program works now.

return True

return call_back
self.fig.canvas.draw()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to use draw_idle which defers the actually drawing until the GUI decides to really repaint the screen.

@tacaswell
Copy link
Member

There is some trailing whitespace in this file that the pep8 checker is catching.

@Thomas00010111
Copy link
Author

Changed to draw_idle() and found the white space.
Draw_idle() does not seem to make a big change in my case. Also the documentation does not contain a lot of information. Can you elaborate a bit more on draw_idle()?

@Thomas00010111
Copy link
Author

It seems I found a difference between draw_idle() and draw(). In my project I cannot use draw_idle(). I have a process (a quadcopter simulator) which generates data, this data should be shown in the graph, realtime would be nice but if the graphs lags behind a bit is not a problem. When I use draw_idle() the graph gets update at once when the simulation process is finished. So I cannot see the quadcopter moving.

@WeatherGod
Copy link
Member

that shouldn't happen, something else must be wrong to cause behavior like
that. The difference between draw() and draw_idle() is simply that the
draw() command forces a paint action from the GUI right then and there,
whether the GUI is ready for it or not. draw_idle() tells the GUI to do a
paint action at the next available moment, and can consolidate multiple
queued draw requests into one. The time difference between these actions is
typically on the order of milliseconds -- at worst hundreds of milliseconds.

Which backend are you using that you are seeing this behavior?

On Thu, Jun 30, 2016 at 6:49 PM, Thomas00010111 notifications@github.com
wrote:

It seems I found a difference between draw_idle() and draw(). In my
project I cannot use draw_idle(). I have a process (a quadcopter simulator)
which generates data, this data should be shown in the graph, realtime
would be nice but if the graphs lags behind a bit is not a problem. When I
use draw_idle() the graph gets update at once when the simulation process
is finished. So I cannot see the quadcopter moving.


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#6637 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AARy-Hhl-gpi6Dfqp5ikW5QGDxEeVjGlks5qREfrgaJpZM4I9Pm_
.

@tacaswell
Copy link
Member

For draw_idle to work the gui event loop needs to be spinning (I also suspect that pan/zoom and the coordinates for the mouse over also do not work when the simulation is running?).

All of the backends should have a process_events which will spin the gui event loop in a blocking way.

I have started to write this stuff up in #4779

@tacaswell tacaswell added this to the 2.0 (style change major release) milestone Jul 1, 2016
@tacaswell tacaswell modified the milestones: 2.0.1 (next bug fix release), 2.0 (style change major release) Jul 1, 2016
@QuLogic QuLogic modified the milestones: 2.0.1 (next bug fix release), 2.0.2 (next bug fix release) May 3, 2017
@tacaswell tacaswell modified the milestones: 2.2 (next next feature release), 2.0.3 (next bug fix release) Jul 11, 2017
@tacaswell tacaswell self-assigned this Jul 11, 2017
@anntzer
Copy link
Contributor

anntzer commented Jan 5, 2018

Mostly obsoleted by #9900; the rest is in #10168.
Thanks for the PR!

@anntzer anntzer closed this Jan 5, 2018
@QuLogic QuLogic modified the milestones: needs sorting, v2.2.0 Feb 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants