syscalls: move away from deprecated CloneElementAt #810

myitcv · 2018-04-29T13:40:15Z

We currently use Node 6.2.2 in CI for our gopherjs test runs.

Ever since the start of the Node 6.x series, v8::Array::CloneElementAt has been deprecated; here's an example from our most recent master build:

make: Entering directory `/home/ubuntu/gopherjs/node-syscall/build'
  CXX(target) Release/obj.target/syscall/syscall.o
../syscall.cc: In function ‘intptr_t toNative(v8::Local<v8::Value>)’:
../syscall.cc:28:51: warning: ‘v8::Local<v8::Object> v8::Array::CloneElementAt(uint32_t)’ is deprecated (declared at /home/ubuntu/.node-gyp/6.2.2/include/node/v8.h:3009): Cloning is not supported. [-Wdeprecated-declarations]
       native[i] = toNative(array->CloneElementAt(i));
                                                   ^

Per #809, in Node 10.x they have finally removed this method and hence attempts to follow the syscall steps fail with:

make: Entering directory '/home/norunners/poc/vert/src/github.com/gopherjs/gopherjs/node-syscall/build'
  CXX(target) Release/obj.target/syscall/syscall.o
../syscall.cc: In function ‘intptr_t toNative(v8::Local<v8::Value>)’:
../syscall.cc:28:35: error: ‘class v8::Array’ has no member named ‘CloneElementAt’
       native[i] = toNative(array->CloneElementAt(i));
                                   ^~~~~~~~~~~~~~

This PR therefore drops our usage of the deprecated method, instead preferring to use v8::Object::Clone() as required.

My C++ skills are however sorely lacking... so perhaps @neelance can offer some thoughts here (given I assume he originated the code?)

We also bump to using Node 10.x in the CI which brings us up-to-date and helps to verify the fix (all the tests pass).

Fixes #809

myitcv · 2018-04-29T19:47:24Z

@neelance - would appreciate if you had a couple of minutes spare to look this over.

@shurcooL @hajimehoshi - this PR is also ready for review.

hajimehoshi · 2018-04-29T19:55:02Z

I'm not familiar with node details, so I'd like to defer to other reviewers on this :-)

dmitshur

Thanks for working on this! Just a few comments for things I could spot.

Does anyone know where the syscall.cc code comes from? Is there an "upstream" version of it? Knowing if that's the case could make reviewing this easier.

dmitshur · 2018-04-29T22:01:26Z

node-syscall/syscall.cc

@@ -25,7 +25,17 @@ intptr_t toNative(Local<Value> value) {
    Local<Array> array = Local<Array>::Cast(value);
    intptr_t* native = reinterpret_cast<intptr_t*>(malloc(array->Length() * sizeof(intptr_t))); // TODO memory leak
    for (uint32_t i = 0; i < array->Length(); i++) {
+#if (NODE_MODULE_VERSION >= NODE_6_0_MODULE_VERSION)
+      v8::Local<v8::Value> elem = array->Get(i);


This file does using namespace v8; on top, and 4 lines above we have Local<Array> array instead of v8::Local<v8::Array> array, etc.

So I think it'd be better to be consistent and drop the namespace here as well:

Local<Value> elem = array->Get(i);

Similarly on another line below.

dmitshur · 2018-04-29T22:02:07Z

node-syscall/syscall.cc

@@ -25,7 +25,17 @@ intptr_t toNative(Local<Value> value) {
    Local<Array> array = Local<Array>::Cast(value);
    intptr_t* native = reinterpret_cast<intptr_t*>(malloc(array->Length() * sizeof(intptr_t))); // TODO memory leak
    for (uint32_t i = 0; i < array->Length(); i++) {
+#if (NODE_MODULE_VERSION >= NODE_6_0_MODULE_VERSION)


What's the difference between NODE_MODULE_VERSION (here) and NODE_MAJOR_VERSION (above, on line 10)? Is it better to use NODE_MODULE_VERSION here?

I honestly don't know; rather pattern matching when it comes to C++... I've switched to using NODE_MAJOR_VERSION

myitcv · 2018-04-30T05:50:14Z

Does anyone know where the syscall.cc code comes from? Is there an "upstream" version of it? Knowing if that's the case could make reviewing this easier.

No, I don't, hence why I pinged @neelance in case he had a couple of minutes to review.

I've just pushed up a couple of fixes in response to your comments. Given the tests pass, which make fairly heavy use of this layer, I think we can be fairly confident the fix is "good enough".

neelance · 2018-04-30T11:20:19Z

I wrote the syscall.cc, there is no upstream.

Is it even necessary to do the Clone?

myitcv · 2018-04-30T11:25:34Z

Is it even necessary to do the Clone?

I'm really unsure; I would defer to you on that question 😄

I've pushed up accaa71 in an attempt to see whether the clone is required or not, at least against the reference that is our CI test suite.

neelance · 2018-04-30T11:26:58Z

I honestly don't know without looking into it some more.

theclapp · 2018-05-20T03:22:06Z

FWIW I dig into syscall.cc and see no reason to use a Clone(). Get() seems fine. My C++ is pretty rusty, but it looks like Get() returns a Local<Value>, which boils down to a *Value. No reason to clone it.

Node.js 10.5.0 is the latest current version according to https://nodejs.org/en/download/current/.

dmitshur

Thanks for the investigation and the fix. Sorry about the delay.

I've tested it and didn't find any issues. LGTM.

CI tests are all passing, except:

$ diff -u <(echo -n) <(git status --porcelain)
--- /dev/fd/63	2018-06-28 18:30:03.118797187 +0000
+++ /dev/fd/62	2018-06-28 18:30:03.118797187 +0000
@@ -0,0 +1 @@
+ M package-lock.json

diff -u <(echo -n) <(git status --porcelain) returned exit code 1

That's not happening on master.

dmitshur · 2018-06-28T18:49:46Z

The diff I get after running npm install locally is:

diff --git a/package-lock.json b/package-lock.json
index ed79b7a..6118aad 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -21,8 +21,8 @@
       "integrity": "sha512-r+MU0rfv4L/0eeW3xZrd16t4NZfK8Ld4SWVglYBb7ez5uXFWHuVRs6xCTrf1yirs9a4j4Y27nn7SRfO6v67XsQ==",
       "dev": true,
       "requires": {
-        "commander": "2.13.0",
-        "source-map": "0.6.1"
+        "commander": "~2.13.0",
+        "source-map": "~0.6.1"
       }
     }
   }

dmitshur · 2018-06-28T18:59:22Z

I guess I'll go with npm install --no-save as a solution. We don't want npm install to be modifying dependencies in package-lock.json in CI.

We don't want npm install to be saving to dependencies and modifying package-lock.json in CI. That should only be done by developers locally. Reference: https://docs.npmjs.com/cli/install.

dmitshur · 2018-06-28T19:40:30Z

Well, that didn't work, because Circle CI runs a second npm install without the --no-save option as part of its "inference commands":

We don't want the inferred npm install command to run without --no-save.

Reference: https://circleci.com/docs/1.0/language-go/#dependencies.

Node.js 10.0.5 comes with a newer npm which causes package-lock.json to be modified after npm install runs. Just use 10.0.0 for now where that doesn't happen.

dmitshur · 2018-06-28T20:55:25Z

Ended up going back to Node.js 10.0.0 which doesn't cause package-lock.json to get modified, it's the simplest solution for now. We'll deal with newer Node.js versions after first migrating off the deprecated Circle CI 1.0 (#766).

govendor fetch +vendor +modified This pulls in gopherjs/gopherjs#810 for gopherjs to fix the build.

Move away from deprecated CloneElementAt

6674f0f

myitcv mentioned this pull request Apr 29, 2018

Failed to Run node-gyp rebuild Needed for gopherjs test on Linux #809

Closed

dmitshur reviewed Apr 29, 2018

View reviewed changes

Address feedback

429cce2

Try without clone

accaa71

bzub mentioned this pull request Apr 30, 2018

Always call Render() in the persistent component hexops/vecty#200

Merged

CI: Update to latest Node.js version.

7117a81

Node.js 10.5.0 is the latest current version according to https://nodejs.org/en/download/current/.

dmitshur approved these changes Jun 28, 2018

View reviewed changes

CI: Use --no-save option for npm install of dev dependencies.

ceaf481

We don't want npm install to be saving to dependencies and modifying package-lock.json in CI. That should only be done by developers locally. Reference: https://docs.npmjs.com/cli/install.

dmitshur added 4 commits June 28, 2018 15:54

CI: Override dependency commands.

e1ea8f1

We don't want the inferred npm install command to run without --no-save.

CI: Add missing inference commands.

0988e64

Reference: https://circleci.com/docs/1.0/language-go/#dependencies.

CI: Change directory into project directory.

1d1026e

CI: Revert back to Node.js 10.0.0.

84e890a

Node.js 10.0.5 comes with a newer npm which causes package-lock.json to be modified after npm install runs. Just use 10.0.0 for now where that doesn't happen.

dmitshur merged commit 5232e51 into gopherjs:master Jun 28, 2018

ralimi added a commit to google/chrome-ssh-agent that referenced this pull request Nov 3, 2018

Update vendored files.

eee7725

govendor fetch +vendor +modified This pulls in gopherjs/gopherjs#810 for gopherjs to fix the build.

syscalls: move away from deprecated CloneElementAt #810

syscalls: move away from deprecated CloneElementAt #810

Uh oh!

Conversation

myitcv commented Apr 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

myitcv commented Apr 29, 2018

Uh oh!

hajimehoshi commented Apr 29, 2018

Uh oh!

dmitshur left a comment

Choose a reason for hiding this comment

Uh oh!

dmitshur Apr 29, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

myitcv Apr 30, 2018

Choose a reason for hiding this comment

Uh oh!

dmitshur Apr 29, 2018

Choose a reason for hiding this comment

Uh oh!

myitcv Apr 30, 2018

Choose a reason for hiding this comment

Uh oh!

myitcv commented Apr 30, 2018

Uh oh!

neelance commented Apr 30, 2018

Uh oh!

myitcv commented Apr 30, 2018

Uh oh!

neelance commented Apr 30, 2018

Uh oh!

theclapp commented May 20, 2018

Uh oh!

dmitshur left a comment

Choose a reason for hiding this comment

Uh oh!

dmitshur commented Jun 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dmitshur commented Jun 28, 2018

Uh oh!

dmitshur commented Jun 28, 2018

Uh oh!

dmitshur commented Jun 28, 2018

Uh oh!

Uh oh!

myitcv commented Apr 29, 2018 •

edited

Loading

dmitshur Apr 29, 2018 •

edited

Loading

dmitshur commented Jun 28, 2018 •

edited

Loading