Improved MEX Compilation #173

jjangsangy · 2015-03-30T09:15:35Z

Continuation of #172

Changes

Only relevant commits to messenger package are included.
Docstrings revised to fit scipy spec
Use platform.machine().endswith('64') over sys.maxsize
Added unit tests to messenger/tests
Tests ensure Matlab architecture is the same as compiled messenger binary.

…l builds

arokem · 2015-04-06T20:50:06Z

messenger/__main__.py

+    -------
+    Namespace containing parsed arguments
+    """
+    parser = ArgumentParser(prog='messenger')


Should this be 'build-messenger'? It might be confusing that this is called 'messenger'

Makes sense, should we rename the entire module to be build-messenger, or just this section of the code?

…l builds

intensifies

Decided to start implementing the big merge with setuptools over current tools consolidated in the messenger module. Hooking into setuptools allows for automatic discovery of requirements and the ability to fine tune almost every aspects of the build process.

arokem · 2015-04-24T16:12:06Z

setup.py

+        import nose
+        args = 'nosetests -v --exe --with-cov '
+        if sys.version_info == (2, 7):
+            args += '--cover-package pymatbridge'


Coverage report should not be mandatory on some versions. Instead it should be an option. That is

python setup.py test --with-coverage

should achieve this effect. Is that possible to do with this?

Notice also that the coverage report on travis seems to be doing something wrong. We are getting a coverage report on some elements of IPython, instead of pymatbridge. Could you take a look at that?

We need a .coveragerc, see the one from scikit-image

arokem · 2015-04-24T16:34:27Z

A couple of comments:

I tried installing this using pip from github on a separate env. It seems to install fine and import finem but I am getting Undefined function 'matlabserver' for input arguments of type 'char'.
The tests pass, but I think that's because they don't get installed properly. Coverage on the pymatbridge module is down to 19%.

jjangsangy · 2015-04-25T00:00:31Z

So after scratching my head for an hour or so, I think I might have figured it out.

My MANIFEST.in was wrong. I don't know how much of this may have contributed to the issue, but it may be the reason why Undefined function 'matlabserver' for input arguments of type 'char'. error might have arisen.
For the tests to run there are several dependencies I did not consider.
- ipython[all] # We need ipython notebook for matlab_magic tests.
- numpy
- wheel # In order to install numpy binary

jjangsangy · 2015-04-25T00:36:50Z

Okay after making these changes and running under virtualenv, heres my coverage report

Name                             Stmts   Miss  Cover   Missing
--------------------------------------------------------------
pymatbridge                         11      4    64%   7-8, 12-13
pymatbridge.__version__              3      0   100%
pymatbridge.compat                   7      2    71%   6-7
pymatbridge.matlab_magic            95      8    92%   35-37, 40, 43, 77, 102, 166-167
pymatbridge.messenger                2      0   100%
pymatbridge.messenger.make         106     69    35%   14-15, 36-87, 97, 112-125, 130-134, 148-164, 193, 199, 203
pymatbridge.messenger.settings      15      1    93%   7
pymatbridge.publish                 49      0   100%
pymatbridge.pymatbridge            233     36    85%   57-58, 80-83, 217, 228, 255-256, 287, 296, 319, 326, 334-343, 388, 396, 445, 496-498, 502, 505-509, 512, 554
--------------------------------------------------------------
TOTAL                              521    120    77%
----------------------------------------------------------------------

The complete coverage is still going down, but this is primarily due to the fact that the messenger is now incorporated into pymatbridge.

arokem · 2015-04-25T01:31:04Z

Did you push these changes to your master branch? I'm not getting them in
a pull.

On Fri, Apr 24, 2015 at 5:36 PM, Sang Han notifications@github.com wrote:

Okay after making these changes, heres my coverage report

Name Stmts Miss Cover Missing

pymatbridge 11 4 64% 7-8, 12-13
pymatbridge.version 3 0 100%
pymatbridge.compat 7 2 71% 6-7
pymatbridge.matlab_magic 95 8 92% 35-37, 40, 43, 77, 102, 166-167
pymatbridge.messenger 2 0 100%
pymatbridge.messenger.make 106 69 35% 14-15, 36-87, 97, 112-125, 130-134, 148-164, 193, 199, 203
pymatbridge.messenger.settings 15 1 93% 7
pymatbridge.publish 49 0 100%

pymatbridge.pymatbridge 233 36 85% 57-58, 80-83, 217, 228, 255-256, 287, 296, 319, 326, 334-343, 388, 396, 445, 496-498, 502, 505-509, 512, 554

TOTAL 521 120 77%

The complete coverage is still going down, but this is primarily due to
the fact that the messenger is now incorporated into pymatbridge.

—
Reply to this email directly or view it on GitHub
#173 (comment)
.

jjangsangy · 2015-04-25T04:01:30Z

Doing it now, there's some code that I had to deactivate in this build. But let's see if I can replicate the coverage report under travis!

jjangsangy · 2015-04-25T07:20:20Z

So strange, for some reason now I'm getting error when running this under a python2.7 virtualenv

error: file '/Users/jjangsangy/Projects/python-matlab-bridge/s' does not exist

Have no idea where in the code is trying to read a file called s.

Can't seem to figure this one out. I think I'm going to sleep on this one and try to tackle it again in the morning.

…p.py

jjangsangy · 2015-04-28T07:01:39Z

Okay, now I was able to replicate the issue on Travis.

error: file '/Users/jjangsangy/Projects/python-matlab-bridge/s' does not exist

It looks like it is actually one of the build scripts running on pymatbridge side of things that seems to be causing the strange formatting issue.

This issue has become such a pain that I'm actually considering starting over from scratch from one of working builds in order to get things running again.

arokem · 2015-04-28T14:53:05Z

I am not sure I entirely understand what "from one of working builds"
means, but if you do start over and break down this PR to several
step-by-step PRs, instead of all at once, it will be easier to review and
merge. If you don't mind breaking all this work up into stages, I think
that it will be easier for everyone.

On Tue, Apr 28, 2015 at 12:01 AM, Sang Han notifications@github.com wrote:

Okay, now I was able to replicate the issue on Travis.

error: file '/Users/jjangsangy/Projects/python-matlab-bridge/s' does not exist

It looks like it is actually one of the build scripts running on
pymatbridge side of things that seems to be causing the strange
formatting issue.

This issue has become such a pain that I'm actually considering starting
over from scratch from one of working builds in order to get things running
again.

—
Reply to this email directly or view it on GitHub
#173 (comment)
.

jjangsangy · 2015-05-06T12:11:18Z

Sure, closing this PR then.

jjangsangy added 24 commits March 25, 2015 06:40

building messenger call now from root

741ff1b

deprecated local.cfg

0ab4034

added get_matlab_env function

b548443

function for fetching zmq source

ba6854a

Fixed windows path

4dfaae1

Fixed windows path

cc4bfef

commit

df41971

Fixed windows path

be0e585

fixed bug with zip-safe

0266b48

cleanup

d7797ee

urllib can go die

d4476c7

get_config() now actually returns a dict for the host platform

436ae0d

oops forgot a comma

7eb892a

fixed bug with zip-safe

f701877

Fixed windows path

a3735b4

get_config() now actually returns a dict for the host platform

5fca4de

Use platform.machine().endswith(64) over sys.maxsize

3551b76

fixed docstring to fit scipy spec

1316635

Mereged messenger commits to master

b70d06f

added tests for messenger functions

5ebb8f6

python 2 glob issue

18cb0cc

python 2 strings do not have maxsplit kwarg, use string module split

ce25c87

tests validate that builds are possible rather than anticipate for al…

d3f605a

…l builds

small bug in get_matlab_dir()

95115f9

arokem mentioned this pull request Apr 6, 2015

Add updated compiled MATLAB messenger mex #178

Merged

arokem reviewed Apr 6, 2015
View reviewed changes

jjangsangy added 12 commits April 23, 2015 12:02

python 2 glob issue

901fe1f

python 2 strings do not have maxsplit kwarg, use string module split

7f329b8

tests validate that builds are possible rather than anticipate for al…

b0b0205

…l builds

small bug in get_matlab_dir()

45a4274

octave build works

8c560f0

and the epic rebasing continues

83ffc35

intensifies

import only bare minimum through exec

1071a1a

working build on osx

321ab0e

setup

6442b60

holy crap it works

3301167

rebase

c92d1de

arokem reviewed Apr 24, 2015
View reviewed changes

Fixed issue with coverage

668ad80

jjangsangy added 2 commits April 27, 2015 23:25

setup.cfg metadata is deprecated, so moved information back into setu…

155a196

…p.py

Go back to running nosetests for now

21e7f28

jjangsangy closed this May 6, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Improved MEX Compilation #173

Improved MEX Compilation #173

Uh oh!

jjangsangy commented Mar 30, 2015

Uh oh!

arokem Apr 6, 2015

Uh oh!

jjangsangy Apr 9, 2015

Uh oh!

arokem Apr 24, 2015

Uh oh!

blink1073 Apr 24, 2015

Uh oh!

arokem commented Apr 24, 2015

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

arokem commented Apr 25, 2015

Name Stmts Miss Cover Missing

pymatbridge.pymatbridge 233 36 85% 57-58, 80-83, 217, 228, 255-256, 287, 296, 319, 326, 334-343, 388, 396, 445, 496-498, 502, 505-509, 512, 554

TOTAL 521 120 77%

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

jjangsangy commented Apr 28, 2015

Uh oh!

arokem commented Apr 28, 2015

Uh oh!

jjangsangy commented May 6, 2015

Uh oh!

Uh oh!

Improved MEX Compilation #173

Improved MEX Compilation #173

Uh oh!

Conversation

jjangsangy commented Mar 30, 2015

Changes

Uh oh!

arokem Apr 6, 2015

Choose a reason for hiding this comment

Uh oh!

jjangsangy Apr 9, 2015

Choose a reason for hiding this comment

Uh oh!

arokem Apr 24, 2015

Choose a reason for hiding this comment

Uh oh!

blink1073 Apr 24, 2015

Choose a reason for hiding this comment

Uh oh!

arokem commented Apr 24, 2015

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

arokem commented Apr 25, 2015

Name Stmts Miss Cover Missing

pymatbridge.pymatbridge 233 36 85% 57-58, 80-83, 217, 228, 255-256, 287, 296, 319, 326, 334-343, 388, 396, 445, 496-498, 502, 505-509, 512, 554

TOTAL 521 120 77%

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

jjangsangy commented Apr 25, 2015

Uh oh!

jjangsangy commented Apr 28, 2015

Uh oh!

arokem commented Apr 28, 2015

Uh oh!

jjangsangy commented May 6, 2015

Uh oh!

Uh oh!