Skip to content

Lapack > 3.5 link issues, deprecated functions #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jgoppert opened this issue Feb 13, 2017 · 19 comments
Closed

Lapack > 3.5 link issues, deprecated functions #8

jgoppert opened this issue Feb 13, 2017 · 19 comments

Comments

@jgoppert
Copy link
Contributor

This will be painful. Scipy just had to go through it as well. We might actually have to try to read the Fortran 77 code to remove dgegs and replace it with dgess etc.

@roryyorke
Copy link
Collaborator

Is the Scipy issue 5266 ? It's refers to Lapack 3.6.0, which lists deprecated functions

CGEGS   CGEGV   CGELSX  CGEQPF  CGGSVD
CGGSVP  CLAHRD  CLATZM  CTZRQF
DGEGS   DGEGV   DGELSX  DGEQPF  DGGSVD
DGGSVP  DLAHRD  DLATZM  DTZRQF
SGEGS   SGEGV   SGELSX  SGEQPF  SGGSVD
SGGSVP  SLAHRD  SLATZM  STZRQF
ZGEGS   ZGEGV   ZGELSX  ZGEQPF  ZGGSVD
ZGGSVP  ZLAHRD  ZLATZM  ZTZRQF

If I grep for those with grep -nHi "\(CGEGS\|CGEGV\|CGELSX\|CGEQPF\|CGGSVD\|CGGSVP\|CLAHRD\|CLATZM\|CTZRQF\|DGEGS\|DGEGV\|DGELSX\|DGEQPF\|DGGSVD\|DGGSVP\|DLAHRD\|DLATZM\|DTZRQF\|SGEGS\|SGEGV\|SGELSX\|SGEQPF\|SGGSVD\|SGGSVP\|SLAHRD\|SLATZM\|STZRQF\|ZGEGS\|ZGEGV\|ZGELSX\|ZGEQPF\|ZGGSVD\|ZGGSVP\|ZLAHRD\|ZLATZM\|ZTZRQF\)" *.f, I get

-*- mode: compilation; default-directory: "~/src/slycot/slycot/src/" -*-
Compilation started at Mon Feb 13 19:58:34

sh find_deprecated.sh
AB08NX.f:202:      EXTERNAL          DLAPMT, DLARFG, DLASET, DLATZM, DORMQR, DORMRQ,
AB08NX.f:311:               CALL DLATZM( 'L', RO+1, MNU-I1, ABCD(IROW+1,I1), 1, T,
AB09CX.f:472:C                DGEQP3, DTZRZF, DTZRQF, DORMQR, and DORMRZ.
AB8NXZ.f:208:     $                  ZLATZM, ZUNMQR, ZUNMRQ
AB8NXZ.f:318:               CALL ZLATZM( 'L', RO+1, MNU-I1, ABCD(IROW+1,I1), 1,
AG08BD.f:241:C     call to the LAPACK Library routines DGEGV or DGGEV.
AG08BY.f:269:     $                   DLATZM, DORMQR, DROT, DSWAP, MB03OY, XERBLA
AG08BY.f:401:            CALL DLATZM( 'L', RO+1, MNR-ICOL, ABCD(IROW+1,ICOL), 1, T,
AG08BZ.f:245:C     call to the LAPACK Library routines ZGEGV or ZGGEV.
AG8BYZ.f:277:     $                   ZLARTG, ZLASET, ZLATZM, ZROT, ZSWAP, ZUNMQR
AG8BYZ.f:409:            CALL ZLATZM( 'L', RO+1, MNR-ICOL, ABCD(IROW+1,ICOL), 1,
MB02QD.f:217:C     modification of the LAPACK routine DGELSX.
MB04GD.f:105:C     Based on LAPACK Library routines DGEQPF and DGERQ2.
MB04NY.f:90:C     Based on LAPACK routines DLARFX and DLATZM.
MB04OY.f:87:C     Based on LAPACK routines DLARFX and DLATZM.
SB01BY.f:125:      EXTERNAL          DLANV2, DLARFG, DLASET, DLASV2, DLATZM, DROT
SB01BY.f:151:            CALL DLATZM( 'Left', M, N, B(1,2), N, TAU1, F(1,1), F(2,1),
SB01BY.f:188:         CALL DLATZM( 'Right', N-1, M, B(1,2), N, TAU1, B(2,1), B(2,2),
SB01BY.f:325:     $   CALL DLATZM( 'Left', M-1, N, B(2,3), N, TAU2, F(2,1), F(3,1),
SB01BY.f:327:      CALL DLATZM( 'Left', M, N, B(1,2), N, TAU1, F(1,1), F(2,1), M,
SB01FY.f:131:      EXTERNAL          DLARFG, DLASET, DLATZM, DROTG, DTRTRI, MA02AD,
SB01FY.f:183:            CALL DLATZM( 'Left', M, N-1, F(2,1), 1, TEMP, F(1,2),
SB04OD.f:416:      EXTERNAL          DCOPY, DGEGS, DGEMM, DGEMV, DLABAD, DLACPY,
SB04OD.f:566:            CALL DGEGS( 'Vectors left', 'Vectors right', M, A, LDA, D,
SB04OD.f:622:            CALL DGEGS( 'Vectors left', 'Vectors right', N, B, LDB, E,
SB06ND.f:186:      EXTERNAL          DCOPY, DGEMM, DLACPY, DLARFG, DLASET, DLATZM,
SB06ND.f:253:               CALL DLATZM( 'Right', JCUR-1, KCUR+1, F(1,JCUR), 1,
SB06ND.f:257:               CALL DLATZM( 'Right', N, KCUR+1, F(1,JCUR), 1,
SB06ND.f:294:                  CALL DLATZM( 'Left', KCUR+1, N-JCUR+1, F(1,JKCUR), 1,
SB06ND.f:309:               CALL DLATZM( 'Left', KCUR+1, M-J+1, F(1,JKCUR), 1,
SG03AD.f:217:C                   DGEGS has failed to converge;
SG03AD.f:395:      EXTERNAL          DCOPY, DGEGS, DLACON, MB01RD, MB01RW, SG03AX,
SG03AD.f:495:         CALL DGEGS( 'Vectors', 'Vectors', N, A, LDA, E, LDE, ALPHAR,
SG03BD.f:206:C                   DGEGS has failed to converge;
SG03BD.f:475:      EXTERNAL          DCOPY, DGEGS, DGEMM, DGEMV, DGEQRF, DGERQF,
SG03BD.f:562:         CALL DGEGS( 'Vectors', 'Vectors', N, A, LDA, E, LDE, ALPHAR,
TB01MD.f:191:      EXTERNAL          DLARFG, DLASET, DLATZM, XERBLA
TB01MD.f:265:         CALL DLATZM( 'Left', NJ+1, N, B(PAR3,PAR1), 1, DZ, A(PAR2,1),
TB01MD.f:267:         CALL DLATZM( 'Right', N, NJ+1, B(PAR3,PAR1), 1, DZ, A(1,PAR2),
TB01MD.f:274:            CALL DLATZM( 'Right', N, NJ+1, B(PAR3,PAR1), 1, DZ,
TB01MD.f:282:            CALL DLATZM( 'Left', NJ+1, PAR4-PAR3+1, B(PAR3,PAR1), 1, DZ,
TB01MD.f:317:         CALL DLATZM( 'Left', NJ+1, PAR6-PAR5+1, A(PAR3,PAR1), 1, DZ,
TB01MD.f:319:         CALL DLATZM( 'Right', N, NJ+1, A(PAR3,PAR1), 1, DZ,
TB01MD.f:326:            CALL DLATZM( 'Right', N, NJ+1, A(PAR3,PAR1), 1, DZ,
TB01ND.f:198:      EXTERNAL          DLARFG, DLASET, DLATZM, XERBLA
TB01ND.f:272:         CALL DLATZM( 'Left', NJ+1, N, C(PAR1,PAR3), LDC, DZ, A(PAR2,1),
TB01ND.f:274:         CALL DLATZM( 'Right', N, NJ+1, C(PAR1,PAR3), LDC, DZ,
TB01ND.f:281:            CALL DLATZM( 'Right', N, NJ+1, C(PAR1,PAR3), LDC, DZ,
TB01ND.f:289:            CALL DLATZM( 'Right', PAR4-PAR3+1, NJ+1, C(PAR1,PAR3), LDC,
TB01ND.f:326:            CALL DLATZM( 'Left', NJ+1, N, A(PAR1,PAR3), LDA, DZ,
TB01ND.f:328:            CALL DLATZM( 'Right', PAR6-PAR5+1, NJ+1, A(PAR1,PAR3), LDA,
TB01ND.f:335:               CALL DLATZM( 'Right', N, NJ+1, A(PAR1,PAR3), LDA, DZ,

Compilation finished at Mon Feb 13 19:58:35

Not all of those are calls (some in comments, etc.), but still it's more than just dgegs :(

@jgoppert jgoppert changed the title Lapack 3.5 link issues deprecated functions Lapack 3.6 link issues deprecated functions Feb 13, 2017
@jgoppert jgoppert changed the title Lapack 3.6 link issues deprecated functions Lapack > 3.5 link issues deprecated functions Feb 13, 2017
@jgoppert jgoppert changed the title Lapack > 3.5 link issues deprecated functions Lapack > 3.5 link issues, deprecated functions Feb 13, 2017
@roryyorke
Copy link
Collaborator

Follow-on thoughts: currently Slycot, if installed via conda, depends on @cwrowley's lapack, which is 3.5.0, so in a sense there's no hurry to fix this.

However, it's possible, and currently quite easy, to link Slycot against mkl (branch, Travis results). If using mkl required fixing these deprecation issues, switching to mkl would become an expensive effort.

@jgoppert
Copy link
Contributor Author

I use conda forge which has lapack 3.6.1:

jgoppert@jmg-bnb1:~/git/control$ conda search lapack
Fetching package metadata .........
clapack                      3.2.1                         0  conda-forge     
lapack                       3.6.0                         0  conda-forge     
                             3.6.1                         0  conda-forge     
                          *  3.6.1                         1  conda-forge  

@jgoppert
Copy link
Contributor Author

This actually spurred me to do this: sympy/sympy#12181

It is working pretty well so far. Having a symbolic control toolbox is pretty powerful for system identification etc. Let me know if anyone is interested in fleshing it out with me.

@slivingston
Copy link
Member

How is that related to this issue? Are you proposing to avoid dependence on LAPACK by performing all operations symbolically?

@jgoppert
Copy link
Contributor Author

For what I am using state space realizations for from python control it makes more sense but I was too lazy to do it until lapack broke slicot. I thought it might be of interest to others. Definitely not a quick solution to do it in sympy, so not a replacement for what broke, but you can follow params through the transformation which is useful.

@jgoppert
Copy link
Contributor Author

Sorry if it was a bit off topic.

@repagh
Copy link
Member

repagh commented Jun 8, 2017

I did DGEGS -> DGGES on repagh/Slycot for now
Had a look at replacing ZLATZM, but that one is still a mystery for me.

@repagh
Copy link
Member

repagh commented Jun 8, 2017

Would it be possible to add a copy of DLATZM to slycot? Renamed?
As far as i can tell, DGEGS -> DGGES and the few ZLATZM -> ZUNMRZ are OK to do, but
the way DLATZM has been used in some places looks pretty fishy to me, and not compatible with the replacement DORMRZ. I have been doing a few, but am reverting that, but I am hitting stuff in SB06ND.f where I think the DLATZM has been used in ways not foreseen by the lapack contributers.

@roryyorke
Copy link
Collaborator

I imagine we could have a lapack-deprecated library within Slycot for this. It seems likely LAPACK's licensing is compatible, though we'd have to check.

Why do you want to use a renamed function?

Are you also trying to build against conda-forge's lapack 3.6.1? It looks to me like MKL is still including these deprecated functions---is building against that an option for you?

Replacing the deprecated functions with their undeprecated counterparts is obviously ideal, but it would be good to have a solid test suite to make sure we haven't broken anything in the process.

I wonder what the other open source SLICOT users (octave-control, Scilab) are doing about this.

@repagh
Copy link
Member

repagh commented Jun 9, 2017

I am trying to get slycot+control into conda-forge.

I found that the deprecated functions are still in my Mac OSX version of anaconda with conda-forge, which I installed earlier and updated. But my Linux version, installed yesterday, did not have the deprecated functions.

We have our students currently using python(x,y) when on windows, and macports python when on Mac OSX, but want to switch to anaconda for all next year. If conda-forge and anaconda have libraries and packages for all three major OSes, we could offer students a more uniform environment -- and finally step to python 3.

I think the licenses are compatible, and I think I will just add the to-be-obsoleted DLATZM from lapack to the slycot sources, re-name the function (as requested also in LAPACK license) to prevent clashes with LAPACK libraries that still have that function, and use that one, so slycot will both compile against 3.6 and lower LAPACK.

It has been a while since I looked at Scilab. Don't know which version of LAPACK they are using.

@repagh
Copy link
Member

repagh commented Jun 9, 2017

Looked at Scilab right now. They are using only part of slicot

http://forge.scilab.org/index.php/p/sCiMake/source/tree/master/scilab-5.2.0/modules/cacsd/src/slicot

@roryyorke
Copy link
Collaborator

Thanks, had a look there; none of the SLICOT files in Scilab are using any deprecated LAPACK functions. Using the list I generated before, the only hits are mb04ny.f and mb04oy.f, and they only mention DLATZM in the comments.

@repagh
Copy link
Member

repagh commented Jun 9, 2017

We are probably not using a number of these files either, only the stuff in the .pyf files gets used.
After some snags (needed to specify the numpy version for building with conda), I have got a well-behaving slycot.

Next step is control.....

@repagh
Copy link
Member

repagh commented Jun 9, 2017

I think I have got it. Found that I had to change the size of the work vector for sg03ad, but all python-control tests ran after that.

Will be testing some more and then create a pull request?

@repagh
Copy link
Member

repagh commented Jun 10, 2017

As far as I can tell, I fixed/updated Slycot to work with newer LAPACK.
I tried to create a pull request on python-control/Slycot, but I somehow can't do that. Through github web I cannot find python-control in the pulldown, and using git it somehow does not work either.

Any hints?

slycotpatches.zip

@repagh
Copy link
Member

repagh commented Jun 10, 2017

I simply copied my changes to patches, deleted my slycot fork (which was from avventi, the original repository), re-forked from python-control, and had no problem making the pull request after that.

@jgoppert
Copy link
Contributor Author

@repagh Thanks so much for doing this! We took over maintenance because you guys were inactive for awhile, but if you want it back I'm sure there would be no problem with that.

@murrayrm
Copy link
Member

Addressed in PR #13.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants