Skip to content

[FakeTensor] Supplement the relevant logic for converting conv1d to conv2d in meta_conv #160408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[FakeTensor] Supplement the relevant logic for converting conv1d to conv2d in meta_conv #160408

wants to merge 1 commit into from
+29 −0

Conversation

thenumberouscode
Copy link
Contributor

@thenumberouscode thenumberouscode commented Aug 12, 2025 •
edited
Loading

Fixes #159462

summary

the issue is caused by the wrong stride of conv1d's result generated by meta_conv:

pytorch/torch/_meta_registrations.py

Lines 2453 to 2471 in 4d5b3f2

shape_out = calc_conv_nd_return_shape(
input_tensor,
weight,
stride,
padding,
dilation,
is_transposed,
groups,
output_padding if is_transposed else None,
)
input_channels_dim = 1
output_channels_dim = 1
if input_tensor.size(input_channels_dim) == 0:
shape_out[output_channels_dim] = 0
out = input_tensor.new_empty(shape_out)
out = out.to(memory_format=pick_memory_format()) # type: ignore[call-overload]
return out

and the wrong stride will be used to codegen size assert in inductor:

pytorch/torch/_inductor/ir.py

Lines 6152 to 6163 in 4d5b3f2

def codegen_size_asserts(self, wrapper: PythonWrapperCodegen) -> None:
if config.size_asserts and not V.graph.cpp_wrapper:
# comparing strides for 0 size tensor is tricky. Ignore them for now.
if sympy_product(self.get_size()) == 0:
return
size = V.graph.wrapper_code.codegen_shape_tuple(self.get_size())
stride = V.graph.wrapper_code.codegen_shape_tuple(self.get_stride())
op_name = self.get_op_name()
wrapper.writeline(
f"assert_size_stride({self.get_name()}, {size}, {stride}, {op_name!r})"
)

reason

So why the computed stride is wrong in the meta_conv function? because the corresponding backend will convert conv1d to conv2d and change the input tensor' size and memory_format(channel last). but the meta_conv do not do this transformation, so a mismatch happend.

pytorch/aten/src/ATen/native/Convolution.cpp

Lines 1502 to 1510 in 4d5b3f2

// Expand 1d -> 2d.
// This is only done for backends that don't natively support 1d spatial input.
if (k == 3 && !input.is_mkldnn() && !input.is_xpu()) {
// avoid accidentally going through NHWC for permuted 3d input.
input = input.contiguous();
params.view1d_as_2d();
input = view4d(input);
weight = view4d(weight);
}

just add corresponding logic in meta_conv.

cc @eellison

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Aug 12, 2025 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/160408

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 28a839d with merge base fea7e9d (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@thenumberouscode
Copy link
Contributor Author

@eellison Please review my PR and leave any comments if you are convenient.

@thenumberouscode thenumberouscode changed the title [FakeTensor] Supplement the relevant logic for converting convert1d t… [FakeTensor] Supplement the relevant logic for converting conv1d to conv2d in meta_conv Aug 12, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
module: inductor open source
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Inductor Fails on Conv1D After Permute with Stride Mismatch Error
2 participants
@thenumberouscode @pytorchbot