Pythonic API #63

hbristow · 2014-04-07T09:49:34Z

This pull request introduces a new pythonic API to Matlab. The functionality is experimental at the moment. If you like this direction, let me know and I can solidify the implementation and remove deprecated code. Don't merge this pull request in its current state. I want to know what you think before I spend any sizable amount of time on it.

Features

Matlab functions are Python functions

# call matlab functions
matlab.sqrt(5) --> 2.23606797749979
matlab.min([5, 4]) --> 4

# bind matlab functions to local variables
sqrt = matlab.sqrt

# matlab functions are real functions
callable(matlab.sqrt) --> True
type(matlab.sqrt) --> instancemethod
print(matlab.sqrt) --> <bound method Matlab.sqrt (verified)>

This allows us to use a bunch of builtin Matlab functionality natively rather than rewrite it ourselves

# get version info
matlab.version() --> '7.13.0.564 (R2011b)'

# run a script
matlab.run('/path/to/script.m')

# call a function not currently on the matlab path
matlab.addpath('/path/to/func')
matlab.func(args)

# evaluate an arbitrary expression
matlab.eval('svd([5 4; 2 1])')

Keyword Arguments

Functions that take optional named arguments using Matlab's name, value hack (plotting, etc) can use Python's native keyword arguments syntax

matlab.plot(x, y, '--', LineWidth=2, MarkerSize=10)

Number of Output Arguments

The behaviour of many Matlab functions changes depending on how many output arguments are provided (seriously, who thought of that...). An option nout argument can be supplied to indicate how many outputs are desired

matlab.min([5, 4]) --> 4
matlab.min([5, 4], nout=2) --> (4, 2)

Native Python docstrings

Help is provided via Python's builtin help() function.

help(matlab.sqrt)

docstrings are cached, so they are only fetched from Matlab once.

Exceptions

Matlab exceptions are captured and re-raised as Python exceptions to indicate errors

# non-existent function
matlab.asdkfjhasdlfjn() --> raises AttributeError: 'Matlab' object has no attribute 'asdkfjhasdlfjn'

# bad inputs
matlab.min() --> raises RuntimeError: MATLAB:minrhs: Not enough input arguments.

If a non-existent function name is called, that function name is blacklisted so subsequent attempts to call that function immediately throw an exception.

Give it a try!

And let me know what you think

arokem · 2014-04-08T01:57:57Z

pymatbridge/matlab/matlabserver.m

+                messenger('respond', 'unrecognized command');
+        end
+
+    catch exception


Yeah - good call on that

arokem · 2014-04-08T02:11:02Z

Wow. This is very cool! I am playing around with this, and it's quite slick. There are still a few things that I wonder about, but generally this is awesome and I think that many people will enjoy using this. Bottom line - I would be happy to integrate this, if you want to work in this direction. It's definitely not in any kind of conflict with anything we are doing. Thanks a lot for your work on this!

arokem · 2014-04-08T02:12:24Z

BTW - what is supposed to happen when you call:

matlab.plot([1,2,3])

or somesuch? I just get back the plot handle, but no plot appears. I imagine that I could save the plot to file using the handle, though I haven't tried that yet.

hbristow · 2014-04-08T02:13:31Z

the desire would be to show the plot and return the handle, as per Matlab. I'll have to check on that. (obviously you'd have to be running a graphical session)

hbristow · 2014-04-08T02:14:17Z

I'll send you my latest changes this evening which should hopefully solidify things a bit more. Is there anything in particular you want me to work/focus on?

arokem · 2014-04-08T02:21:53Z

Great. Not sure about direction to focus on - I'll play around with this a
little bit more.

On Mon, Apr 7, 2014 at 7:14 PM, Hilton Bristow notifications@github.comwrote:

I'll send you my latest changes this evening which should hopefully
solidify things a bit more. Is there anything in particular you want me to
work/focus on?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-39805146
.

hbristow · 2014-04-08T02:22:53Z

Do you have a Json extension to serialize numpy arrays?

arokem · 2014-04-08T02:26:44Z

No - maybe we can somehow use the zmq messenger and the pyzmq API and
special case np arrays?

See:
http://zeromq.github.io/pyzmq/serialization.html

We can probably assume that a numpy array on the python side is a matrix on
the matlab side and vice versa. Maybe except in the case of strings :-) Oh,
matlab...

On Mon, Apr 7, 2014 at 7:22 PM, Hilton Bristow notifications@github.comwrote:

Do you have a Json extension to serialize numpy arrays?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-39805569
.

arokem · 2014-04-08T02:27:16Z

pymatbridge/pymatbridge.py

+        return resp['result']
+
+    @property
+    def __doc__(self, parent):


Does this work for you? I keep getting

TypeError: __doc__() takes exactly 2 arguments (1 given)

Must have broken it when I changed to using weakrefs. Will put it on my TODO

Which reminds me: don't forget to write some tests for this API. Just to be
sure that things are solid moving forward. If you need help with that, let
me know.

On Mon, Apr 7, 2014 at 7:30 PM, Hilton Bristow notifications@github.comwrote:

In pymatbridge/pymatbridge.py:

}

if nout:

req['nout'] = nout

resp = parent()._execute(req)

if not resp['success']:

if resp['result'] == 'MATLAB:UndefinedFunction':

parent().blacklist.add(self.name)

delattr(parent(), self.name)

raise AttributeError(attribute_msg.format(self.name))

raise RuntimeError(resp['result'] +': '+ resp['message'])

else:

self.**name** = self.name + ' (verified)'

return resp['result']

@Property

def doc(self, parent):

Must have broken it when I changed to using weakrefs. Will put it on my
TODO

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63/files#r11374470
.

arokem · 2014-04-08T03:42:40Z

OK - I am looking at this a bit more, and ran the test suite. Does the added functionality really have to break the previous functionality?

arokem · 2014-04-08T03:42:58Z

Sorry - didn't mean to close it. Hit the wrong button...

hbristow · 2014-04-08T03:46:50Z

Hey, no it doesn't need to break the old API. We can discuss deprecation techniques. I just haven't focussed on that yet.

hbristow · 2014-04-08T03:53:04Z

Is there a particular reason you're using distutils rather than setuptools in setup.py. Also, is version.py used for anything other than including in setup.py, and do you really need to include it with exec?

arokem · 2014-04-08T03:53:14Z

Cool - got it.

On Mon, Apr 7, 2014 at 8:46 PM, Hilton Bristow notifications@github.comwrote:

Hey, no it doesn't need to break the old API. We can discuss deprecation
techniques. I just haven't focussed on that yet.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-39809183
.

arokem · 2014-04-08T03:56:29Z

No particular reason for any of these decisions, except inertia.

On Mon, Apr 7, 2014 at 8:53 PM, Hilton Bristow notifications@github.comwrote:

Is there a particular reason you're using distutils rather than setuptoolsin
setup.py. Also, is version.py used for anything other than including in
setup.py, and do you really need to include it with exec?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-39809400
.

hbristow · 2014-04-08T03:57:07Z

okay. Same for using "%" rather than "format()" for formatting strings?

arokem · 2014-04-08T03:58:23Z

That's actually a more deeply entrenched personal preference. I think it
parses better.

On Mon, Apr 7, 2014 at 8:57 PM, Hilton Bristow notifications@github.comwrote:

okay. Same for using "%" rather than "format()" for formatting strings?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-39809549
.

hbristow · 2014-04-08T03:59:00Z

okay, I won't touch any of that

hbristow · 2014-04-08T04:02:23Z

out of curiosity, what have you currently been using pymatbridge for?

arokem · 2014-04-08T04:04:49Z

Mostly interleaving calls to large legacy code-bases in Matlab into ipython
notebooks that then call visualizations/calculations written in python. My
main interest is the notebook and matlab magic. I've heard from others that
are using it in similar manner.

On Mon, Apr 7, 2014 at 9:02 PM, Hilton Bristow notifications@github.comwrote:

out of curiosity, what have you currently been using pymatbridge for?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-39809755
.

…d by Matlab

…b.drawnow(). TODO: Allow the EDT to run while zmq is blocking for a request

…ped minor version number to signify new API

hbristow · 2014-04-10T06:38:28Z

Hey, what framework do you use for testing? I see that your test cases don't subclass unittest.TestCase, so how do you discover and run tests? Do you just rely on the smarts of nose?

arokem · 2014-04-10T12:40:22Z

Indeed. We use nose

On Wed, Apr 9, 2014 at 11:38 PM, Hilton Bristow notifications@github.comwrote:

Hey, what framework do you use for testing? I see that your test cases
don't subclass unittest.TestCase, so how do you discover and run tests?
Do you just rely on the smarts of nose?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-40047428
.

Mailerdaimon · 2014-04-14T08:49:00Z

This looks like a great addition! Is this ready for testing or should i wait? I have no problems with a few bugs here and there which i would happily report or fix.

hbristow · 2014-04-14T12:33:11Z

Hey!
I haven't had any time to write a compatibility interface for the old API
yet. I'll try and get to it over Easter (along with writing some tests).
H

On Mon, Apr 14, 2014 at 6:48 PM, Martin notifications@github.com wrote:

This looks like a great addition! Is this ready for testing or should i
wait? I have no problems with a few bugs here and there which i would
happily report or fix.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-40344820
.

arokem · 2014-05-10T21:57:54Z

Hey @hbristow - have you had a chance to take a look? If you don't have time to look at this, would you mind rebasing this on top of master, so I could try to work on this?

Thanks!

hbristow · 2014-05-26T00:00:09Z

Hey!
Sorry, I've been writing papers at the moment, and haven't had any time to
work on fun things. I'll make sure to rebase it for you tomorrow.
H

On Sun, May 11, 2014 at 7:57 AM, Ariel Rokem notifications@github.comwrote:

Hey @hbristow https://github.com/hbristow - have you had a chance to
take a look? If you don't have time to look at this, would you mind
rebasing this on top of master, so I could try to work on this?

Thanks!

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-42755563
.

arokem · 2014-05-28T16:58:17Z

Alright - now that we have a new release - time to take a deeper look at this! I have a new branch for this up here:

https://github.com/arokem/python-matlab-bridge/tree/hbristow-feat-pythonic

Rebased on top of the new master and ready to start breaking stuff!

haoxingz · 2014-05-28T17:04:20Z

Hi Ariel,

Congratulations on the new release! I think I'll pick this one and start to
look at it.

At the same time I've been reading about the IPython's messaging mechanism
and it is very similar to what we have now, although they are richer in
functionality. I'll take the "kernel" as a long time project and keep
thinking about it. I'll keep you updated on the progress.

Best,
Haoxing

2014-05-28 9:58 GMT-07:00 Ariel Rokem notifications@github.com:

Alright - now that we have a new release - time to take a deeper look at
this! I have a new branch for this up here:

https://github.com/arokem/python-matlab-bridge/tree/hbristow-feat-pythonic

Rebased on top of the new master and ready to start breaking stuff!

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/63#issuecomment-44435176
.

nzjrs · 2014-08-05T13:40:17Z

Just a note. I extended this branch (#77) with more flexible handling of variables such that they need not be serialized and returned to Python every call.

I think I also fixed the problem with adjacent matrix arguments being coalesced (7d38469)

blink1073 · 2015-02-18T03:21:48Z

Implemented in #145.

Initial pythonic API teaser

12e0411

arokem reviewed Apr 8, 2014
View reviewed changes

pymatbridge/matlab/matlabserver.m

messenger('respond', 'unrecognized command');

end

catch exception

Copy link

Owner

arokem Apr 8, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah - good call on that

arokem reviewed Apr 8, 2014
View reviewed changes

arokem closed this Apr 8, 2014

arokem reopened this Apr 8, 2014

Hilton Bristow added 3 commits April 8, 2014 22:49

streamlined API

a8cd5cc

Added 2D numpy (de-)serialization support. 3D+ currently being mangle…

95e8abb

…d by Matlab

Improve socket reboot

4e80438

Hilton Bristow added 3 commits April 9, 2014 23:37

Multi-dimensional numpy array support

1ba4829

Allow graphical plots by default. Draw the plots with a call to matla…

de4d9b9

…b.drawnow(). TODO: Allow the EDT to run while zmq is blocking for a request

Migrated setup.py to setuptools. Merged version.py into setup.py. Bum…

07acc4e

…ped minor version number to signify new API

arokem mentioned this pull request Aug 19, 2014

Unify variable visibility and optionally leave in MATLAB #77

Closed

blink1073 mentioned this pull request Feb 17, 2015

Add function maker #145

Merged

blink1073 closed this Feb 18, 2015

Pythonic API #63

Pythonic API #63

Uh oh!

Conversation

hbristow commented Apr 7, 2014

Features

Matlab functions are Python functions

Keyword Arguments

Number of Output Arguments

Native Python docstrings

Exceptions

Give it a try!

Uh oh!

arokem Apr 8, 2014

Choose a reason for hiding this comment

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

arokem Apr 8, 2014

Choose a reason for hiding this comment

Uh oh!

hbristow Apr 8, 2014

Choose a reason for hiding this comment

Uh oh!

arokem Apr 8, 2014

Choose a reason for hiding this comment

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

hbristow commented Apr 8, 2014

Uh oh!

arokem commented Apr 8, 2014

Uh oh!

hbristow commented Apr 10, 2014

Uh oh!

arokem commented Apr 10, 2014

Uh oh!

Mailerdaimon commented Apr 14, 2014

Uh oh!

hbristow commented Apr 14, 2014

Uh oh!

arokem commented May 10, 2014

Uh oh!

hbristow commented May 26, 2014

Uh oh!

arokem commented May 28, 2014

Uh oh!

haoxingz commented May 28, 2014

Uh oh!

nzjrs commented Aug 5, 2014

Uh oh!

blink1073 commented Feb 18, 2015