cc @gaogaotiantian

I'm not able to reproduce this on either main or 3.13. Could you make a full example?

[1, 2] and breakpoint()

> /home/gaogaotiantian/programs/mycpython/scrabble.py(1)<module>()
-> [1, 2] and breakpoint()
(Pdb) list
  1  -> [1, 2] and breakpoint()
[EOF]
(Pdb) $_frame.f_lineno
1
(Pdb) 

Ok took some time to find how to reproduce, but turns out you need at least one extra statement after the breakpoint call. So something like:

[1,2] and breakpoint()

a = 1

should work.

Ah, interesting. Worked for me, thanks!

@iritkatriel the bytecode of

[1, 2] and breakpoint()
a = 2

is

   0           RESUME                   0

   1           LOAD_SMALL_INT           1
               LOAD_SMALL_INT           2
               BUILD_LIST               2
               COPY                     1
               TO_BOOL
               POP_JUMP_IF_FALSE        7 (to L1)
               POP_TOP
               LOAD_NAME                0 (breakpoint)
               PUSH_NULL
               CALL                     0

  --   L1:     POP_TOP

   2           LOAD_SMALL_INT           2
               STORE_NAME               1 (a)
               LOAD_CONST               0 (None)
               RETURN_VALUE

The label L1 is the short circuit label. Is it expected that the bytecode is not associated with any line number? I mean I can understand that the label itself is probably not generated directly by the code, just want to confirm if this is a desired behavior. If so, I'll need to work on some workaround for pdb.

The POP_TOP is there because we just compiled an expression (and nothing is consuming its result from the stack). If the POP_TOP was only reachable via fall through, the line number of the previous instruction would propagate to it. But here it's reachable via a jump as well (and it seems we don't check that the jump was on the same line as the fall through, but even if we did this would fail in the case where the boolean expression spans more than one line).

There are situations (such as return and raise instructions) where we force instructions to have a line number even if it requires making multiple copies of some instructions. But this is not one of those cases. Why is it a problem here that there is no line number?

Why is it a problem here that there is no line number?

Because breakpoint() will stop at the next instruction after CALL. In this case, it would be POP_TOP. pdb relies on the line number for multiple things, for example, show the current line. It will also use it for list command to show source code. It would be helpful to know the line number.

The workaround could be stop at opcode that has line number, but in this case, the next instruction belongs to the next line, which would break the behavior that breakpoint() will stop at the line of breakpoint().

Can the breakpoint builtin do something to help know where it is?

Well it also affects pdb.set_trace() so we can't only deal with it in breakpoint(). If assigning a line number to this opcode is a no-go, we probably have 3 ways to deal with this:

1. Let it be, it's a corner case anyway.
2. Make a proxy to "fake" the line number. We can intercept the access to f.f_lineno inside pdb and fake some number. The way I can think of is to get the previous opcode (hopefully CALL) and the line number associated. The downside of this way is that we might have plenty of places that we need to patch and the magic is not pretty. Also when the users access $_frame, they'll see that the lineno is -1, which contradicts what we show them.
3. Like I said, for set_trace(), stop only when there is a line number. In this case, it will stop at the next line, which is a slight deviation from our existing behavior.

As this is a rare case, I don't think either of the options above is horrible (although I'm a bit hesitated about 2. as it impacts too much code in pdb). However, from the other point of view, none of the options is perfect either. My perfect solution was for the opcode to have a line number, but maybe that's too much a burden for the compiler.

My perfect solution was for the opcode to have a line number, but maybe that's too much a burden for the compiler.

It's not that it's a burden on the compiler. It's that line numbers are static, so if you can reach this NOP from two different paths, you need to make two copies of this NOP, and then add a jump over one of the copies to the next instruction (here it's the LOAD_SMALL_INT). So we emit more code, and add a jump, just for this.

It might be possible to improve the situation if the compiler does propagate the line number if it's the same for both predecessors (as it is in this example). It won't completely fix the problem, this will still not work:

>>> (1 or
...   2) and breakpoint()

Yeah I understand the problem behind. I actually have a question here:

If the compiler knows the return value is not used, why does it need to copy the return value? Can it do a simple POP_JUMP_IF_FALSE? We can save a COPY, TO_BOOL and the extra POP_TOP. (I'm not the expert in bytecodes so I might be completely wrong).

Also, even if we want a single POP_TOP at the end, do we leave the line number empty? Can we think of it as "pop the eval value from stack after the full expression" so the opcode should be associated to the end of the expression (last line)? I think we have this kind of idea in mind when we deal with loops or other code blocks? I mean in reality, the opcodes we generated are not directly corresponding to an exact line of code, is it better to just leave it blank? Comparing to assigning a reasonable but not completely accurate value? In my mind it gives more useful information to users, even not considering the usage of pdb.

I think line numbers are majorly used by debuggers, so it's debugging information anyway (correct me if I'm wrong). "I don't know where this opcode is generated" is correct by less useful information for debugging.

Yeah I understand the problem behind. I actually have a question here:

If the compiler knows the return value is not used, why does it need to copy the return value? Can it do a simple POP_JUMP_IF_FALSE? We can save a COPY, TO_BOOL and the extra POP_TOP. (I'm not the expert in bytecodes so I might be completely wrong).

It's a feature of the language that the result of a boolean expression is the value of the element that gave the truthy result:

>>> x = 0 or 1 or 2
>>> x
1

It's common to use this for None handling:

def f(arg = None):
    arg = arg or DEFAULT_VALUE_OF_ARG

Also, even if we want a single POP_TOP at the end, do we leave the line number empty? Can we think of it as "pop the eval value from stack after the full expression" so the opcode should be associated to the end of the expression (last line)?

The last line of the expression may not have executed. we want the POP_TOP to have the line number of the last instruction from the expression that executed, otherwise it would mess up the trace.

I think line numbers are majorly used by debuggers, so it's debugging information anyway (correct me if I'm wrong). "I don't know where this opcode is generated" is correct by less useful information for debugging.

They are also used for other things, like tracebacks and coverage. Coverage is particularly sensitive to inaccurate line numbers.

>>> x = 0 or 1 or 2
>>> x
1

These are cases where the expressions are assigned to a variable, and this is not causing issue if I understood correctly. I meant for the cases where the expressions are not assigned to anything, so the value is unused. We should be able to optimize that case right? It's faster and it generates less code (and it's rare).

otherwise it would mess up the trace.

Hmm okay it might even generate a new line event which is probably something we don't want. I guess I'll have to deal with this in pdb.

This is fixed in main and backported to 3.13.