Skip to content

[ci] Add riscv opt-int build #143979

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

zhangfeiv0
Copy link
Contributor

@zhangfeiv0 zhangfeiv0 commented Dec 30, 2024

Hi, @malfet
Based on the previous discussion:

RISCV CI support · Issue #141550 · pytorch/pytorch

I have cross-compiled PyTorch for the RISC-V architecture on x86_64 Ubuntu 24.04 and created a new PR for it. Could you please help review it?

cc @malfet @seemethere

@zhangfeiv0 zhangfeiv0 requested review from a team and jeffdaily as code owners December 30, 2024 08:23
Copy link

pytorch-bot bot commented Dec 30, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/143979

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 52dea10 with merge base 33a1996 (image):

UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@pytorch-bot pytorch-bot bot added the release notes: releng release notes category label Dec 30, 2024
@colesbury colesbury requested a review from malfet December 30, 2024 14:48
@colesbury colesbury added module: build Build system issues triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module module: risc-v All issues related to RISC-V architecture labels Dec 30, 2024
@zhangfeiv0
Copy link
Contributor Author

Hi, @malfet

Here’s a gentle ping: we have fixed some code errors detected by the CI. When you have time, could you please take a look and provide any suggestions?

@zhangfeiv0
Copy link
Contributor Author

@Skylion007 Could you help review it if you have time?

@wangfanv0
Copy link

@malfet @Skylion007

@malfet
Copy link
Contributor

malfet commented Feb 20, 2025

@zhangfeiv0 Do you mind changing it to opt-in only for now and delete binary wheels workflow? Let's see if builds are passing first and than we can take next steps.

@malfet
Copy link
Contributor

malfet commented Feb 20, 2025

@pytorchbot rebase

Copy link

pytorch-bot bot commented Feb 20, 2025

Unknown label ciflow/riscv64.
Currently recognized labels are

  • ciflow/binaries
  • ciflow/binaries_libtorch
  • ciflow/binaries_wheel
  • ciflow/inductor
  • ciflow/inductor-periodic
  • ciflow/inductor-rocm
  • ciflow/inductor-perf-compare
  • ciflow/inductor-micro-benchmark
  • ciflow/inductor-micro-benchmark-cpu-x86
  • ciflow/inductor-cu126
  • ciflow/linux-aarch64
  • ciflow/mps
  • ciflow/nightly
  • ciflow/periodic
  • ciflow/rocm
  • ciflow/s390
  • ciflow/slow
  • ciflow/trunk
  • ciflow/unstable
  • ciflow/xpu
  • ciflow/torchbench
  • ciflow/autoformat

@malfet
Copy link
Contributor

malfet commented Feb 20, 2025

@pytorchbot rebase

@pytorchmergebot
Copy link
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

Successfully rebased riscv_ci_support onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout riscv_ci_support && git pull --rebase)

@pytorchmergebot
Copy link
Collaborator

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

@pytorchmergebot
Copy link
Collaborator

Tried to rebase and push PR #143979, but it was already up to date. Try rebasing against main by issuing:
@pytorchbot rebase -b main

@malfet
Copy link
Contributor

malfet commented Feb 20, 2025

Making some changes to the workflow now. If it finished build in reasonable amount of time, than yes, it could be landed as an opt-in for now

@malfet malfet changed the title [ci] Add riscv support [ci] Add riscv opt-int build Feb 20, 2025
@malfet
Copy link
Contributor

malfet commented Feb 21, 2025

@zhangfeiv0, you'll need to adapt your PR to make it work with CI build, i.e. docker for CI is build using files in .ci/docker folder, rather than in .ci/docker/manywheel, and are stored on ECR/ghcr.io rather than on docker hub

@pytorchmergebot
Copy link
Collaborator

Successfully rebased riscv_ci_support onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout riscv_ci_support && git pull --rebase)


export CMAKE_CROSSCOMPILING=TRUE
export CMAKE_SYSTEM_NAME=Linux
export CMAKE_SYSTEM_PROCESSOR=riscv64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO if one really cross-compiles, it should be CMAKE_TARGET_PROCESSOR rather than a system one

@zhangfeiv0
Copy link
Contributor Author

During cross-compilation, some third-party libraries need to run on the host machine, including protocbuf and sleef.

  • Protobuf: As per your suggestion, it was disabled.
  • Sleef: Still requires execution on the host machine during compilation.

To address this, I introduced docker/setup-qemu-action@v3 to emulate the execution of the necessary binaries in these third-party libraries. This approach was also tested in my local environment. However, in the last workflow run, the protobuf binary failed to execute, suggesting this solution did not work as intended.
I moved this step into the Build step to see if it resolves the issue. If successful, this would eliminate the need to disable protobuf.

Additionally, some steps you previously deleted need to be retained; otherwise, compilation errors will still occur during the build phase.

@malfet Could you restart the workflow and check again? Thanks!

@zhangfeiv0 zhangfeiv0 requested a review from malfet April 27, 2025 02:02
@lamaalrajih
Copy link

@malfet bump!

@houseroad
Copy link
Member

Shall we rebase this again? @zhangfeiv0

@zhangfeiv0 zhangfeiv0 force-pushed the riscv_ci_support branch 2 times, most recently from dd57a91 to e820d79 Compare July 19, 2025 00:38
@zhangfeiv0
Copy link
Contributor Author

@houseroad Friendly ping

zhangfeiv0 and others added 12 commits August 4, 2025 13:55
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
Signed-off-by: Zhang fei <zhangfei@iscas.ac.cn>
@zhangfeiv0 zhangfeiv0 requested a review from malfet August 5, 2025 06:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: build Build system issues module: risc-v All issues related to RISC-V architecture open source release notes: releng release notes category triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants