[Bug #19340] Fix bundle gems with test revision #7146

nobu · 2023-01-18T13:00:13Z

Build temporary gem package from cloned repository if test revision is set.

hsbt · 2023-01-19T03:35:07Z

I tested this branch with https://bugs.ruby-lang.org/issues/19340#note-3

1. with the current master and 3. with the following patch are ok.

rbs             3.0.0.dev.1   https://github.com/ruby/rbs 8bd4e46bc87775d457d76a707d54fbe97781d9f2

$ make extract-gems
/opt/homebrew/opt/coreutils/libexec/gnubin/mkdir -p gems/src
Cloning https://github.com/ruby/rbs
Cloning into 'gems/src/rbs'...
remote: Enumerating objects: 21225, done.
remote: Counting objects: 100% (1092/1092), done.
remote: Compressing objects: 100% (467/467), done.
remote: Total 21225 (delta 678), reused 935 (delta 596), pack-reused 20133
Receiving objects: 100% (21225/21225), 5.85 MiB | 3.97 MiB/s, done.
Resolving deltas: 100% (14155/14155), done.
Building rbs@8bd4e46bc87775d457d76a707d54fbe97781d9f2 to rbs-3.0.0.dev.1.gem
From ssh://github.com/ruby/rbs
 * branch              8bd4e46bc87775d457d76a707d54fbe97781d9f2 -> FETCH_HEAD
HEAD is now at 8bd4e46b Merge pull request #1209 from ruby/env-utilities
  Successfully built RubyGem
  Name: rbs
  Version: 3.0.0.dev.1
  File: rbs-3.0.0.dev.1.gem
/opt/homebrew/opt/coreutils/libexec/gnubin/mkdir -p .bundle/gems
Extracting bundle gem minitest-5.17.0...
Unpacked gems/minitest-5.17.0.gem
Extracting bundle gem power_assert-2.0.3...
Unpacked gems/power_assert-2.0.3.gem
Extracting bundle gem rake-13.0.6...
Unpacked gems/rake-13.0.6.gem
Extracting bundle gem test-unit-3.5.7...
Unpacked gems/test-unit-3.5.7.gem
Extracting bundle gem rexml-3.2.5...
Unpacked gems/rexml-3.2.5.gem
Extracting bundle gem rss-0.2.9...
Unpacked gems/rss-0.2.9.gem
Extracting bundle gem net-ftp-0.2.0...
Unpacked gems/net-ftp-0.2.0.gem
Extracting bundle gem net-imap-0.3.4...
Unpacked gems/net-imap-0.3.4.gem
Extracting bundle gem net-pop-0.1.2...
Unpacked gems/net-pop-0.1.2.gem
Extracting bundle gem net-smtp-0.3.3...
Unpacked gems/net-smtp-0.3.3.gem
Extracting bundle gem matrix-0.4.2...
Unpacked gems/matrix-0.4.2.gem
Extracting bundle gem prime-0.1.2...
Unpacked gems/prime-0.1.2.gem
Extracting bundle gem typeprof-0.21.4...
Unpacked gems/typeprof-0.21.4.gem
Extracting bundle gem debug-1.7.1...
Unpacked gems/debug-1.7.1.gem
Extracting bundle gem rbs-3.0.0.dev.1...
Unpacked gems/rbs-3.0.0.dev.1.gem

But 2. case is not working yet.

rbs             2.8.3   https://github.com/ruby/rbs 8bd4e46bc87775d457d76a707d54fbe97781d9f2

$ make extract-gems
/opt/homebrew/opt/coreutils/libexec/gnubin/mkdir -p gems/src
Cloning https://github.com/ruby/rbs
Cloning into 'gems/src/rbs'...
remote: Enumerating objects: 21225, done.
remote: Counting objects: 100% (1092/1092), done.
remote: Compressing objects: 100% (467/467), done.
remote: Total 21225 (delta 678), reused 931 (delta 596), pack-reused 20133
Receiving objects: 100% (21225/21225), 5.85 MiB | 3.50 MiB/s, done.
Resolving deltas: 100% (14154/14154), done.
/opt/homebrew/opt/coreutils/libexec/gnubin/mkdir -p .bundle/gems
Extracting bundle gem minitest-5.17.0...
Unpacked gems/minitest-5.17.0.gem
Extracting bundle gem power_assert-2.0.3...
Unpacked gems/power_assert-2.0.3.gem
Extracting bundle gem rake-13.0.6...
Unpacked gems/rake-13.0.6.gem
Extracting bundle gem test-unit-3.5.7...
Unpacked gems/test-unit-3.5.7.gem
Extracting bundle gem rexml-3.2.5...
Unpacked gems/rexml-3.2.5.gem
Extracting bundle gem rss-0.2.9...
Unpacked gems/rss-0.2.9.gem
Extracting bundle gem net-ftp-0.2.0...
Unpacked gems/net-ftp-0.2.0.gem
Extracting bundle gem net-imap-0.3.4...
Unpacked gems/net-imap-0.3.4.gem
Extracting bundle gem net-pop-0.1.2...
Unpacked gems/net-pop-0.1.2.gem
Extracting bundle gem net-smtp-0.3.3...
Unpacked gems/net-smtp-0.3.3.gem
Extracting bundle gem matrix-0.4.2...
Unpacked gems/matrix-0.4.2.gem
Extracting bundle gem prime-0.1.2...
Unpacked gems/prime-0.1.2.gem
Extracting bundle gem typeprof-0.21.4...
Unpacked gems/typeprof-0.21.4.gem
Extracting bundle gem debug-1.7.1...
Unpacked gems/debug-1.7.1.gem
Extracting bundle gem rbs-2.8.3...
Unpacked gems/rbs-2.8.3.gem

Build temporary gem package from cloned repository if test revision is set.

hsbt · 2023-01-20T07:48:25Z

Current code will clone gems/src/rbs/rbs.gemspec as git repository.

From dc26e060f558556d936c8a91f8b3bd97a5e24768 Mon Sep 17 00:00:00 2001
From: Hiroshi SHIBATA <hsbt@ruby-lang.org>
Date: Fri, 20 Jan 2023 16:45:40 +0900
Subject: [PATCH] gemspec

---
 defs/gmake.mk | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git defs/gmake.mk defs/gmake.mk
index ad397336350..62cae651719 100644
--- defs/gmake.mk
+++ defs/gmake.mk
@@ -330,11 +330,11 @@ $(srcdir)/.bundle/.timestamp:
    $(MAKEDIRS) $@

 define build-gem
-$(srcdir)/gems/src/$(1)/$(1).gemspec: | $(srcdir)/gems/src
+$(srcdir)/gems/src/$(1): | $(srcdir)/gems/src
    $(ECHO) Cloning $(4)
    $(Q) $(GIT) clone $(4) $$(@)

-$(srcdir)/.bundle/.timestamp/$(1).revision: $(srcdir)/gems/src/$(1)/$(1).gemspec \
+$(srcdir)/.bundle/.timestamp/$(1).revision: $(srcdir)/gems/src/$(1) \
    $(if $(if $(wildcard $$(@)),$(filter $(3),$(shell cat $$(@)))),,PHONY) \
    | $$(@D)
    $(ECHO) Update $(1) to $(3)
@@ -344,7 +344,7 @@ $(srcdir)/.bundle/.timestamp/$(1).revision: $(srcdir)/gems/src/$(1)/$(1).gemspec
    :
    echo $(3) | $(IFCHANGE) $$(@) -

-$(srcdir)/gems/$(1)-$(2).gem: $(srcdir)/gems/src/$(1)/$(1).gemspec \
+$(srcdir)/gems/$(1)-$(2).gem: $(srcdir)/gems/src/$(1) \
        $(srcdir)/.bundle/.timestamp/$(1).revision
    $(ECHO) Building $(1)@$(3) to $$(@)
    $(Q) $(BASERUBY) -C "$(srcdir)" \
--
2.39.1

I applied above patch. After that it works correctly with 1-3 cases.

hsbt · 2023-01-20T09:25:34Z

diff --git defs/gmake.mk defs/gmake.mk
index ad397336350..f8e0bf0280e 100644
--- defs/gmake.mk
+++ defs/gmake.mk
@@ -332,7 +332,7 @@ $(srcdir)/.bundle/.timestamp:
 define build-gem
 $(srcdir)/gems/src/$(1)/$(1).gemspec: | $(srcdir)/gems/src
    $(ECHO) Cloning $(4)
-   $(Q) $(GIT) clone $(4) $$(@)
+   $(Q) $(GIT) clone $(4) $$(@D)

 $(srcdir)/.bundle/.timestamp/$(1).revision: $(srcdir)/gems/src/$(1)/$(1).gemspec \
    $(if $(if $(wildcard $$(@)),$(filter $(3),$(shell cat $$(@)))),,PHONY) \

It works fine.

nobu force-pushed the bundled-gems-build branch from 01bfe30 to 963da55 Compare January 19, 2023 11:23

nobu added 2 commits January 20, 2023 14:51

[Bug #19340] Fix bundle gems with test revision

e10d01f

Build temporary gem package from cloned repository if test revision is set.

Use timestamp files to track test revision of bundled gems

216621c

nobu force-pushed the bundled-gems-build branch from 963da55 to 216621c Compare January 20, 2023 05:52

Fixed clone name

b13f5c9

hsbt force-pushed the bundled-gems-build branch from e6d4aca to b13f5c9 Compare January 20, 2023 09:40

Need gemspec files instead of cloned directories

ea962c2

hsbt approved these changes Jan 23, 2023

View reviewed changes

nobu merged commit be6a347 into ruby:master Jan 26, 2023

nobu deleted the bundled-gems-build branch January 26, 2023 03:50

mame mentioned this pull request Jan 26, 2023

Make "make exts" build only gems listed in gems/bundled_gems #7119

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug #19340] Fix bundle gems with test revision #7146

[Bug #19340] Fix bundle gems with test revision #7146

nobu commented Jan 18, 2023

hsbt commented Jan 19, 2023

hsbt commented Jan 20, 2023

hsbt commented Jan 20, 2023

[Bug #19340] Fix bundle gems with test revision #7146

[Bug #19340] Fix bundle gems with test revision #7146

Conversation

nobu commented Jan 18, 2023

hsbt commented Jan 19, 2023

hsbt commented Jan 20, 2023

hsbt commented Jan 20, 2023