Skip to content

Fix gc_verify_internal_consistency error for pattern_matching in ripper #7627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

yui-knk
Copy link
Contributor

@yui-knk yui-knk commented Mar 30, 2023

gc_verify_internal_consistency reports "found internal inconsistency" for "test_pattern_matching.rb".

http://ci.rvm.jp/results/trunk-gc-asserts@ruby-sp2-docker/4501173

Ruby's parser manages objects by two different ways.

  1. For parser
  • markable node holds objects
  • call RB_OBJ_WRITTEN with p->ast as parent
  • mark_ast_value marks objects
  1. For ripper
  • unmarkable node, NODE_RIPPER/NODE_CDECL, holds objects
  • call rb_ast_add_mark_object. This function calls rb_hash_aset then RB_OBJ_WRITTEN is called with mark_hash as parent
  • mark_hash marks objects

However in current pattern_matching implementation

  • markable node holds objects
  • call rb_ast_add_mark_object

This commit fix it to be #2.

This was inconsistency however always mark_hash is made young by rb_ast_add_mark_object call then objects are not collected.

`gc_verify_internal_consistency` reports "found internal inconsistency"
for "test_pattern_matching.rb".

http://ci.rvm.jp/results/trunk-gc-asserts@ruby-sp2-docker/4501173

Ruby's parser manages objects by two different ways.

1. For parser

* markable node holds objects
* call `RB_OBJ_WRITTEN` with `p->ast` as parent
* `mark_ast_value` marks objects

2. For ripper

* unmarkable node, NODE_RIPPER/NODE_CDECL, holds objects
* call `rb_ast_add_mark_object`. This function calls `rb_hash_aset` then
  `RB_OBJ_WRITTEN` is called with `mark_hash` as parent
* `mark_hash` marks objects

However in current pattern_matching implementation

* markable node holds objects
* call `rb_ast_add_mark_object`

This commit fix it to be #2.

This was inconsistency however always `mark_hash` is
made young by `rb_ast_add_mark_object` call then objects
are not collected.
Copy link
Member

@XrXr XrXr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was inconsistency however always mark_hash is made young by rb_ast_add_mark_object call then objects are not collected.

Do I understand correctly that it was a write barrier timing problem?rb_ast_add_mark_object can yield to the GC due to allocation, and previously, when it did yield, the GC observed new out edges from imemo_ast added without write barriers.

@yui-knk
Copy link
Contributor Author

yui-knk commented Mar 31, 2023

I don't think it's a write barrier timing problem.
imemo_ast marks objects held by NODE directly by mark_ast_value but imemo_ast doesn't mark objects held by mark_hash.
If object is assigned to NODE_ARYPTN's nd_rval, the object is marked directly from imemo_ast. But in this case RB_OBJ_WRITTEN is never called. Then inconsistency always happens if imemo_ast is in an old generation.

@yui-knk yui-knk merged commit 3488eda into ruby:master Mar 31, 2023
@yui-knk yui-knk deleted the fix_gc_inconsistency_for_pattern_matching_in_ripper branch March 31, 2023 00:38
k-tsj referenced this pull request Mar 31, 2023
It should also check for duplicate names.
@XrXr
Copy link
Member

XrXr commented Mar 31, 2023

Ah, so previously, the references were added to imemo_ast without any write barriers running. Now the objects are held by mark_hash instead of imemo_ast?

@yui-knk
Copy link
Contributor Author

yui-knk commented Apr 1, 2023

Previously, the references were added to both (1) imemo_ast without write barriers by rb_node_newnode(NODE_ARYPTN, pre_args, rest_arg, post_args, &NULL_LOC); for example (2) mark_hash with write barriers via rb_ast_add_mark_object. Now, yes, mark_hash holds the object and imemo_ast does not hold it.

@XrXr
Copy link
Member

XrXr commented Apr 3, 2023

Thank you for patiently helping me understand!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants