This would be a good topic to raise on the numpy mailing list.

This would be a good topic to raise on the numpy mailing list.

@charris , could you please help with this?

You can subscribe to the mailing list at https://mail.python.org/accounts/signup/?next=/mailman3/lists/numpy-discussion.python.org/. Then just send an email to numpy-discussion@python.org. You can also open a numpy issue with the suggestion.

@HaoZeke does this seem reasonable, or is (for example) LFortran already exposing a more maintainable and robust parser?

@rgommers , sincerely any rich compiler-level technology is irrelevant to this call. The beauty of crackfortran is in its simplicity. Crackfortran parses the source (actually, the interface part only, not the code!) unarmed, only by regex matching. This approach is more oriented on extracting analytics from the syntactically-correct source. Unlike that, compiler parser has to be precise about may possible error states to assist correction of malformed code. Furthermore, extracting a JSON usable by a non-compiler person from a yacc/bison-generated AST would be another tough adventure. So my point is: these are two different worlds. There are cars and carts, and good cars are not obsoleting the joy of carting.

@HaoZeke does this seem reasonable, or is (for example) LFortran already exposing a more maintainable and robust parser?

LFortran has a more robust parser, but it is still less complete than that of f2py at the moment. It will take time. Perhaps more importantly, at the moment the development is such that a dependency on lfortran is quite heavy, and would include LLVM. This could also be fixed, but I am not sure when it would be feasible to split the parser development into a separate project. Perhaps @certik would be able to give more insight.

With regards to @dmikushin's eloquent comment:

... So my point is: these are two different worlds. There are cars and carts, and good cars are not obsoleting the joy of carting.

I actually tend to agree. @pearu has mentioned on several occasions that F2PY doesn't need to penalize the user for not constructing correct Fortran (also F2PY takes the !f2py directives and usercode which LFortran would need a plugin ecosystem to support).

The cleanest route to couple LFortran and F2PY in the near future would be to generate F2PY's crackfortran style input from LFortran's ASR which would then guarantee semantic correctness while keeping the backend from F2PY agnostic to changes. Since LFortran also can generate JSON representations, it would be a good place to start integrating the two in any case.

Some other projects also use crackfortran (e.g. sphinx-fortran) so I guess it could be a good idea to expose it.

However some more thoughts for @dmikushin:

• Would need documentation
• Use cases / examples
• AFAIK we use recursive dictionaries, do they have a good representation in JSON?

In principle, letting users clean up crackfortran's output on their own is a good idea, it could be paired with the hooks @pearu introduced in #19388.

@dmikushin is this still relevant?

I'm still interested!

Just some data points:

• crackfortran is not a Fortran parser, LFortran is. One could say that crackfortran is a Fortran "parser" that parses only Fortran declarations, crackfortran does not parse Fortran execution statements (and never will as these are irrelevant for f2py tasks).
• crackfortran is able to "parse" Fortran 77 (as well as 90 and newer) while LFortran does not support Fortran 77. This (among other things) basically defines why LFortran-based tools cannot replace f2py.

Regarding releasing crackfortran as a standalone tool: I am not sure if it is worth it just for the sake of using crackfortran as a tool. Some parts of crackfortran depend on numpy, so you would need to have numpy available anyway for your software (update: I misunderstood the aim of this PR). Instead, I would recommend reviving https://github.com/pearu/f2py that provides a true Fortran parser in Python that supports both fixed and free formats (hence, also Fortran 77). But be warned, it will require a dedication of more than just a couple of months.

while LFortran does not support Fortran 77.

LFortran supports F77.

while LFortran does not support Fortran 77.

LFortran supports F77.

[OT Warning]
LFortran supports fixed format Fortran which is not the same as F77. For instance, lfortran gives syntax error to

      subroutinefoo
      end

which is a valid Fortran 77 source code. A true Fortran 77 parser cannot rely on nowadays tokenizers that LFortran uses. Hence my claim.
[End of OT Warning]

AFAIK LFortran plans to support all of F77, but is continually in development, so eventually it will get there.

https://github.com/pearu/f2py is a good place to look, but the frontend (CLI) should follow the newer f2pyarg design (from #21923)... I'd be happy to mentor someone working on upgrading the parser in-place, but likely won't have too much time myself in the near future (would be happy to be proven wrong of course).

Practically speaking of course, one of the current highest barriers to major changes within F2PY is the lack of existing test coverage within NumPy and in the larger context of parsing Fortran code.

@dmikushin could you clarify the scope of your proposed work? Was the intent just to move the existing crackfortran code into a callable module? Or is this a discussion from which a larger project is meant to emerge?

For instance, lfortran gives syntax error to

You have to tell LFortran to use its fixed-form parser:

$ lfortran --fixed-form a.f --show-ast
(TranslationUnit
    [(Subroutine
        foo
        []
        []
        ()
        ()
        []
        []
        []
        []
        []
        []
        []
    )]
)

LFortran supports fixed-form, for example it can parse all of SciPy correctly.

If you find any bug, you can report it, but overall LFortran supports F77 and fixed-form. You can contact me offline if you want to check with me before posting online what LFortran can or cannot do.