Skip to content

Include decorators in Class and FunctionDef range #49

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

MichaReiser
Copy link
Contributor

@MichaReiser MichaReiser commented May 17, 2023

This PR changes the ClassDef and FunctionDef definitions to include the range of their decorators.

Example

@test
def f(): 
	pass
  • Before: The functions' range was from line 2, column 0 to line 3, column 9
  • New: The functions' range spans from line 1, column 0 to line 3, column 9 (includes the @test decorator)

Motivation

This change is because these are the only nodes for where the property parent.range.includes(child.range) does not hold. The fact that this property is not respected means it is e.g. impossible to use a binary search over the AST to find a node by its range.

Downsides

The main downside that I'm aware of is that diagnostics using range.start to compute the line index for functions and classes may now incorrectly point to the line of the function's decorator instead of the function definition. I don't know if this is a problem for RustPython. Ruff already uses custom infrastructure to extract the range of the function name only to limit the diagnostic range (and e.g. avoid highlighting the whole function or class including the body)

Ruff

astral-sh/ruff#4467

@youknowone
Copy link
Member

Oh.. this is a Python spec omg.

Python 3.11.2 (main, Feb 16 2023, 02:55:59) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ast
>>> ast.parse("""
... @decorator
... def x(): pass
... """)
<ast.Module object at 0x100738be0>
>>> a = _
>>> f = a.body[0]
>>> f.lineno
3
>>> f.decorator_list[0].lineno
2

We will really need a feature flag for ruff.

@youknowone youknowone mentioned this pull request May 18, 2023
4 tasks
@youknowone
Copy link
Member

youknowone commented May 18, 2023

I'd better to understand it better if this is also related to symbol table or SyntaxError.
If not, it can be handled in ast module with this patch

@youknowone
Copy link
Member

youknowone commented May 18, 2023

@MichaReiser Do you want to make it has different value only for more-attributes feature right now? That has no blocker.

@MichaReiser
Copy link
Contributor Author

@MichaReiser Do you want to make it has different value only for more-attributes feature right now? That has no blocker.

In my view, this stretches feature flags to the limit, or even over the limit. We would need to rename it at least. I also plan to make other range changes:

def a():
	pass

	# comment

The range of the function should include the #comment, which it does not today.

I think maintaining this change in a fork is straightforward enough and probably easier than dealing with even more conditional code that has very limited test coverage.

@youknowone
Copy link
Member

youknowone commented May 18, 2023

The range of the function should include the #comment, which it does not today.

it seems going really different.

I think maintaining this change in a fork is straightforward enough and probably easier than dealing with even more conditional code that has very limited test coverage.

Yes, that sounds like truth.

@youknowone
Copy link
Member

Reusing ast crate but making a fork of parser crate makes sense to me.

@youknowone
Copy link
Member

We can make a new crate like rustpython-parser-util to share fundamental parser utilities except for the details.

@MichaReiser
Copy link
Contributor Author

We can make a new crate like rustpython-parser-util to share fundamental parser utilities except for the details.

I don't think that this is necessary. I still plan to pull in upstream changes. I don't expect to many conflicts since the grammar shouldn't change frequently

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants