ruby/json@cb61a00ba8

* Add buffer protocol

* Modify for some review comments

* Per-object buffer availability

* Rename to MemoryView from Buffer and make compilable

* Support integral repeat count in memory view format

* Support 'x' for padding bytes

* Add rb_memory_view_parse_item_format

* Check type in rb_memory_view_register

* Update dependencies in common.mk

* Add test of MemoryView

* Add test of rb_memory_view_init_as_byte_array

* Add native size format test

* Add MemoryView test utilities

* Add test of rb_memory_view_fill_contiguous_strides

* Skip spaces in format string

* Support endianness specifiers

* Update documentation

* Support alignment

* Use RUBY_ALIGNOF

* Fix format parser to follow the pack format

* Support the _ modifier

* Parse count specifiers in get_format_size function.

* Use STRUCT_ALIGNOF

* Fix test

* Fix test

* Fix total size for the case with tail padding

* Fix rb_memory_view_get_item_pointer

* Fix rb_memory_view_parse_item_format again

We can not call a non-isolated Proc in multiple ractors.

The memory view interface added at 890bc2c is experimental new C-API
set.  This feature permits extension libraries to share a memory area
that contains such a numerical array and a bitmap image.  This is
designed by referring to Python's buffer protocol.
[[Feature #13767]]
[[Feature #14722]]

ruby/readline-ext@f9783c0739

Matz want to try to freeze all Range objects.
[Feature #15504]

to prevent:

```
test/json/json_common_interface_test.rb:182: warning: mismatched indentations at 'end' with 'def' at 169
```

The deprecation warning was disabled, and the code to check the warning
was removed at 996af2c, thus capture_io
is no longer needed.

…- members"

by explicitly setting `Warning[:deprecated] = true`.
I removed "capture_io" at 79063d8, but
it printed the warning when `RUBYOPT=-w`.

This change makes the warnings enabled explicitly, capture and check the
warning.

…t initialized"

`make test-all` was very noisy by warnings like

```
Leaked file descriptor: Net::TestSMTP#test_start_with_position_argument: 6 : #<TCPSocket:fd 6, AF_INET, 127.0.0.1, 43770>
```

Since r63443, `-std=gnu99 -D_XOPEN_SOUCE=x00` is added to Solaris'
`CPPFLAGS`.  `CPPFLAGS` is shared among `CC` / `CXX`.  This results in
both `__STDC_VERSION__` and `__cplusplus` to be defined at the same time
for a C++ compilation, only on Solaris.

It seems the `CPPFLAGS` addition is intentional.  We sould not touch
that part.  Instead we need to reroute this by always check for
`__cplusplus` first.

The added test case crashes the interpreter because it makes
ObjectSpace.internal_class_of return the second VALUE slot of an AST
imemo object. The second VALUE slot of `struct rb_ast_struct` is
not a VALUE and not a pointer to a Ruby object.

* Enhanced RDoc for String#succ

We should let fibers update their own references on compaction.  I don't
think we need the thread to update the associated fiber because there
will be a fiber object on the heap that knows how to update itself.

https://github.com/ruby/ruby/runs/1169621878

Update the lldb script so it can mostly recover a Ruby stack trace from
a core file.  It's still missing line numbers and dealing with CFUNCs,
but you use it like this:

```
(lldb) rbbt ec
rb_control_frame_t TYPE
0x7f6fd6555fa0     EVAL   ./bootstraptest/runner.rb error!!
0x7f6fd6555f68     METHOD ./bootstraptest/runner.rb main
0x7f6fd6555f30     METHOD ./bootstraptest/runner.rb in_temporary_working_directory
0x7f6fd6555ef8     METHOD /home/aaron/git/ruby/lib/tmpdir.rb mktmpdir
0x7f6fd6555ec0     BLOCK  ./bootstraptest/runner.rb block in in_temporary_working_directory
0x7f6fd6555e88     CFUNC
0x7f6fd6555e50     BLOCK  ./bootstraptest/runner.rb block (2 levels) in in_temporary_working_directory
0x7f6fd6555e18     BLOCK  ./bootstraptest/runner.rb block in main
0x7f6fd6555de0     METHOD ./bootstraptest/runner.rb exec_test
0x7f6fd6555da8     CFUNC
0x7f6fd6555d70     BLOCK  ./bootstraptest/runner.rb block in exec_test
0x7f6fd6555d38     CFUNC
0x7f6fd6555d00     TOP    /home/aaron/git/ruby/bootstraptest/test_insns.rb error!!
0x7f6fd6555cc8     CFUNC
0x7f6fd6555c90     BLOCK  /home/aaron/git/ruby/bootstraptest/test_insns.rb block in <top (required)>
0x7f6fd6555c58     METHOD ./bootstraptest/runner.rb assert_equal
0x7f6fd6555c20     METHOD ./bootstraptest/runner.rb assert_check
0x7f6fd6555be8     METHOD ./bootstraptest/runner.rb show_progress
0x7f6fd6555bb0     METHOD ./bootstraptest/runner.rb with_stderr
0x7f6fd6555b78     BLOCK  ./bootstraptest/runner.rb block in show_progress
0x7f6fd6555b40     BLOCK  ./bootstraptest/runner.rb block in assert_check
0x7f6fd6555b08     METHOD ./bootstraptest/runner.rb get_result_string
0x7f6fd6555ad0     METHOD ./bootstraptest/runner.rb make_srcfile
0x7f6fd6555a98     CFUNC
0x7f6fd6555a60     BLOCK  ./bootstraptest/runner.rb block in make_srcfile
```

Getting the main execution context is difficult (it is stored in a
thread local) so for now you must supply an ec and this will make a
backtrace

https://hackerone.com/reports/954433

https://rubyci.org/logs/rubyci.s3.amazonaws.com/rhel8/ruby-master/log/20201016T063003Z.fail.html.gz
```
  1) Error:
Rinda::TestRingServer#test_do_reply:
Timeout::Error: timeout
    /home/chkbuild/chkbuild/tmp/build/20201016T063003Z/ruby/test/rinda/test_rinda.rb:837:in `sleep'
    /home/chkbuild/chkbuild/tmp/build/20201016T063003Z/ruby/test/rinda/test_rinda.rb:837:in `wait_for'
    /home/chkbuild/chkbuild/tmp/build/20201016T063003Z/ruby/test/rinda/test_rinda.rb:659:in `_test_do_reply'
    /home/chkbuild/chkbuild/tmp/build/20201016T063003Z/ruby/test/rinda/test_rinda.rb:643:in `block in test_do_reply'
    /home/chkbuild/chkbuild/tmp/build/20201016T063003Z/ruby/test/rinda/test_rinda.rb:807:in `with_timeout'
    /home/chkbuild/chkbuild/tmp/build/20201016T063003Z/ruby/test/rinda/test_rinda.rb:643:in `test_do_reply'
```

Buggy native extensions could have mark functions that cause stack
overflow. When a stack overflow happens during GC, Ruby used to recover
by raising an exception, which runs the interpreter. It's not safe to
run the interpreter during GC since the GC is in an inconsistent state.
This could cause object allocation during GC, for example.

Instead of running the interpreter and potentially causing a crash down
the line, fail fast and abort.

in addition to de5e8d0

(1) nobody uses it (gem-codesearch)
(2) the data strucuture will be changed.

iv_index_tbl manages instance variable indexes (ID -> index).
This data structure should be synchronized with other ractors
so introduce some VM locks.

This patch also introduced atomic ivar cache used by
set/getinlinecache instructions. To make updating ivar cache (IVC),
we changed iv_index_tbl data structure to manage (ID -> entry)
and an entry points serial and index. IVC points to this entry so
that cache update becomes atomically.

According to the log of ac803ab, I
found that a thread terminates silently due to "recycled object" of
id2ref:

```
"/home/chkbuild/chkbuild/tmp/build/20201017T033002Z/ruby/lib/drb/drb.rb:366:in `_id2ref'"
"/home/chkbuild/chkbuild/tmp/build/20201017T033002Z/ruby/lib/drb/drb.rb:366:in `to_obj'"
"/home/chkbuild/chkbuild/tmp/build/20201017T033002Z/ruby/lib/drb/drb.rb:1528:in `to_obj'"
"/home/chkbuild/chkbuild/tmp/build/20201017T033002Z/ruby/lib/drb/drb.rb:1847:in `to_obj'"
"/home/chkbuild/chkbuild/tmp/build/20201017T033002Z/ruby/lib/drb/drb.rb:1136:in `method_missing'"
"/home/chkbuild/chkbuild/tmp/build/20201017T033002Z/ruby/test/rinda/test_rinda.rb:652:in `block in do_reply'"
```
https://rubyci.org/logs/rubyci.s3.amazonaws.com/rhel8/ruby-master/log/20201017T033002Z.log.html.gz

I believe that this unintentional thread termination has caused
intermittent timeout failure of `TestRingServer#test_do_reply`.

The root cause of the "recycled object" issue is a bug of
`TestRingServer#test_do_reply`.  It creates a callback Proc object but
does not hold the reference to the object:

```
callback = DRb::DRbObject.new callback
```

The original "callback" object is GC'ed unintentionally.
I could consistently reproduce this issue on my machine by adding
`GC.stress = true` at the first of `_test_do_reply` method body.

This change uses another local variable name, "callback_orig", to keep
the original Proc object.

This reverts commit ac803ab.

Remove debugging code that is no longer needed

…o_reply"

This reverts commit de5e8d0.

Remove the debugging code that is no longer needed

To reduce the number of locking for encoding manipulation,
enc_table::list is splited to ::default_list and ::additional_list.
::default_list is pre-allocated and no need locking to access to
the ::default_list. If additional encoding space is needed, use
::additional_list and this list need to use locking.
However, most of case, ::default_list is enough.

  This reverts commit f39021b.

  * Added sync task for digest
  * Update doc/* for default gems
  * Update the latest version of gemspec

To access TLS, it is faster to use language TLS specifier instead
of using pthread_get/setspecific functions.

Original proposal is: Use native thread locals. ruby#3665

This reverts commit de17e2d.

This patch can introduce race condition because of conflicting
read/write access for enc_table::default_list. Maybe we need to
freeze default_list at the end of Init_encdb() in enc/encdb.c.