Skip to content

Commit 51ce346

Browse files
committed
Merge branch 'develop' of https://github.com/explosion/spaCy into develop
2 parents 15de2bb + 2b35bb7 commit 51ce346

File tree

7 files changed

+143
-11
lines changed

7 files changed

+143
-11
lines changed

.github/contributors/uwol.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
# spaCy contributor agreement
2+
3+
This spaCy Contributor Agreement (**"SCA"**) is based on the
4+
[Oracle Contributor Agreement](http://www.oracle.com/technetwork/oca-405177.pdf).
5+
The SCA applies to any contribution that you make to any product or project
6+
managed by us (the **"project"**), and sets out the intellectual property rights
7+
you grant to us in the contributed materials. The term **"us"** shall mean
8+
[ExplosionAI UG (haftungsbeschränkt)](https://explosion.ai/legal). The term
9+
**"you"** shall mean the person or entity identified below.
10+
11+
If you agree to be bound by these terms, fill in the information requested
12+
below and include the filled-in version with your first pull request, under the
13+
folder [`.github/contributors/`](/.github/contributors/). The name of the file
14+
should be your GitHub username, with the extension `.md`. For example, the user
15+
example_user would create the file `.github/contributors/example_user.md`.
16+
17+
Read this agreement carefully before signing. These terms and conditions
18+
constitute a binding legal agreement.
19+
20+
## Contributor Agreement
21+
22+
1. The term "contribution" or "contributed materials" means any source code,
23+
object code, patch, tool, sample, graphic, specification, manual,
24+
documentation, or any other material posted or submitted by you to the project.
25+
26+
2. With respect to any worldwide copyrights, or copyright applications and
27+
registrations, in your contribution:
28+
29+
* you hereby assign to us joint ownership, and to the extent that such
30+
assignment is or becomes invalid, ineffective or unenforceable, you hereby
31+
grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge,
32+
royalty-free, unrestricted license to exercise all rights under those
33+
copyrights. This includes, at our option, the right to sublicense these same
34+
rights to third parties through multiple levels of sublicensees or other
35+
licensing arrangements;
36+
37+
* you agree that each of us can do all things in relation to your
38+
contribution as if each of us were the sole owners, and if one of us makes
39+
a derivative work of your contribution, the one who makes the derivative
40+
work (or has it made will be the sole owner of that derivative work;
41+
42+
* you agree that you will not assert any moral rights in your contribution
43+
against us, our licensees or transferees;
44+
45+
* you agree that we may register a copyright in your contribution and
46+
exercise all ownership rights associated with it; and
47+
48+
* you agree that neither of us has any duty to consult with, obtain the
49+
consent of, pay or render an accounting to the other for any use or
50+
distribution of your contribution.
51+
52+
3. With respect to any patents you own, or that you can license without payment
53+
to any third party, you hereby grant to us a perpetual, irrevocable,
54+
non-exclusive, worldwide, no-charge, royalty-free license to:
55+
56+
* make, have made, use, sell, offer to sell, import, and otherwise transfer
57+
your contribution in whole or in part, alone or in combination with or
58+
included in any product, work or materials arising out of the project to
59+
which your contribution was submitted, and
60+
61+
* at our option, to sublicense these same rights to third parties through
62+
multiple levels of sublicensees or other licensing arrangements.
63+
64+
4. Except as set out above, you keep all right, title, and interest in your
65+
contribution. The rights that you grant to us under these terms are effective
66+
on the date you first submitted a contribution to us, even if your submission
67+
took place before the date you sign these terms.
68+
69+
5. You covenant, represent, warrant and agree that:
70+
71+
* Each contribution that you submit is and shall be an original work of
72+
authorship and you can legally grant the rights set out in this SCA;
73+
74+
* to the best of your knowledge, each contribution will not violate any
75+
third party's copyrights, trademarks, patents, or other intellectual
76+
property rights; and
77+
78+
* each contribution shall be in compliance with U.S. export control laws and
79+
other applicable export and import laws. You agree to notify us if you
80+
become aware of any circumstance which would make any of the foregoing
81+
representations inaccurate in any respect. We may publicly disclose your
82+
participation in the project, including the fact that you have signed the SCA.
83+
84+
6. This SCA is governed by the laws of the State of California and applicable
85+
U.S. Federal law. Any choice of law rules will not apply.
86+
87+
7. Please place an “x” on one of the applicable statement below. Please do NOT
88+
mark both statements:
89+
90+
* [x] I am signing on behalf of myself as an individual and no other person
91+
or entity, including my employer, has or will have rights with respect to my
92+
contributions.
93+
94+
* [ ] I am signing on behalf of my employer or a legal entity and I have the
95+
actual authority to contractually bind that entity.
96+
97+
## Contributor Details
98+
99+
| Field | Entry |
100+
|------------------------------- | -------------------- |
101+
| Name | Ulrich Wolffgang |
102+
| Company name (if applicable) | |
103+
| Title or role (if applicable) | |
104+
| Date | 2017-11-05 |
105+
| GitHub username | uwol |
106+
| Website (optional) | https://uwol.github.io/ |

spacy/about.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# https://github.com/pypa/warehouse/blob/master/warehouse/__about__.py
44

55
__title__ = 'spacy-nightly'
6-
__version__ = '2.0.0a18'
6+
__version__ = '2.0.0a19'
77
__summary__ = 'Industrial-strength Natural Language Processing (NLP) with Python and Cython'
88
__uri__ = 'https://spacy.io'
99
__author__ = 'Explosion AI'

spacy/lang/it/tag_map.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -316,5 +316,5 @@
316316
"V__VerbForm=Ger": {"pos": "VERB"},
317317
"V__VerbForm=Inf": {"pos": "VERB"},
318318
"X___": {"pos": "X"},
319-
"_SP": {"pos": "_SP"}
319+
"_SP": {"pos": "SPACE"}
320320
}

spacy/pipeline.pyx

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -318,7 +318,7 @@ class Tensorizer(Pipe):
318318
loss, d_scores = self.get_loss(docs, golds, scores)
319319
d_inputs = bp_scores(d_scores, sgd=sgd)
320320
d_inputs = self.model.ops.xp.split(d_inputs, len(self.input_models), axis=1)
321-
for d_input, bp_input in zip(d_inputs, bp_inputs):
321+
for d_input, bp_input in zip(d_inputs, bp_inputs):
322322
bp_input(d_input, sgd=sgd)
323323
if losses is not None:
324324
losses.setdefault(self.name, 0.)
@@ -415,7 +415,11 @@ class Tagger(Pipe):
415415
vocab.morphology.assign_tag_id(&doc.c[j], tag_id)
416416
idx += 1
417417
if tensors is not None:
418-
doc.extend_tensor(tensors[i])
418+
if isinstance(doc.tensor, numpy.ndarray) \
419+
and not isinstance(tensors[i], numpy.ndarray):
420+
doc.extend_tensor(tensors[i].get())
421+
else:
422+
doc.extend_tensor(tensors[i])
419423
doc.is_tagged = True
420424

421425
def update(self, docs, golds, drop=0., sgd=None, losses=None):
@@ -777,7 +781,8 @@ class TextCategorizer(Pipe):
777781
def predict(self, docs):
778782
scores = self.model(docs)
779783
scores = self.model.ops.asarray(scores)
780-
return scores
784+
tensors = [doc.tensor for doc in docs]
785+
return scores, tensors
781786

782787
def set_annotations(self, docs, scores, tensors=None):
783788
for i, doc in enumerate(docs):

spacy/syntax/nn_parser.pyx

Lines changed: 13 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -540,7 +540,9 @@ cdef class Parser:
540540
return None
541541

542542
backprops = []
543-
d_tokvecs = state2vec.ops.allocate(tokvecs.shape)
543+
# Add a padding vector to the d_tokvecs gradient, so that missing
544+
# values don't affect the real gradient.
545+
d_tokvecs = state2vec.ops.allocate((tokvecs.shape[0]+1, tokvecs.shape[1]))
544546
cdef float loss = 0.
545547
n_steps = 0
546548
while todo:
@@ -623,7 +625,9 @@ cdef class Parser:
623625
bp_vectors))
624626
else:
625627
backprop_lower.append((ids, d_vector, bp_vectors))
626-
d_tokvecs = self.model[0].ops.allocate(tokvecs.shape)
628+
# Add a padding vector to the d_tokvecs gradient, so that missing
629+
# values don't affect the real gradient.
630+
d_tokvecs = state2vec.ops.allocate((tokvecs.shape[0]+1, tokvecs.shape[1]))
627631
self._make_updates(d_tokvecs, bp_tokvecs, backprop_lower, sgd,
628632
cuda_stream)
629633

@@ -676,7 +680,8 @@ cdef class Parser:
676680
(ids.size, d_state_features.shape[2]))
677681
self.model[0].ops.scatter_add(d_tokvecs, ids,
678682
d_state_features)
679-
bp_tokvecs(d_tokvecs, sgd=sgd)
683+
# Padded -- see update()
684+
bp_tokvecs(d_tokvecs[:-1], sgd=sgd)
680685

681686
@property
682687
def move_names(self):
@@ -746,7 +751,11 @@ cdef class Parser:
746751
for j in range(doc.length):
747752
doc.c[j] = state.c._sent[j]
748753
if tensors is not None:
749-
doc.extend_tensor(tensors[i])
754+
if isinstance(doc.tensor, numpy.ndarray) \
755+
and not isinstance(tensors[i], numpy.ndarray):
756+
doc.extend_tensor(tensors[i].get())
757+
else:
758+
doc.extend_tensor(tensors[i])
750759
self.moves.finalize_doc(doc)
751760

752761
for hook in self.postprocesses:

spacy/tests/parser/test_beam_parse.py

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,22 @@
22
from __future__ import unicode_literals
33

44
import pytest
5+
from ...language import Language
6+
from ...pipeline import DependencyParser
57

68

79
@pytest.mark.models('en')
8-
def test_beam_parse(EN):
10+
def test_beam_parse_en(EN):
911
doc = EN(u'Australia is a country', disable=['ner'])
1012
ents = EN.entity(doc, beam_width=2)
1113
print(ents)
14+
15+
16+
def test_beam_parse():
17+
nlp = Language()
18+
nlp.add_pipe(DependencyParser(nlp.vocab), name='parser')
19+
nlp.parser.add_label('nsubj')
20+
nlp.begin_training()
21+
22+
doc = nlp.make_doc(u'Australia is a country')
23+
nlp.parser(doc, beam_width=2)

spacy/vectors.pyx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -358,7 +358,7 @@ cdef class Vectors:
358358
def load_vectors(path):
359359
xp = Model.ops.xp
360360
if path.exists():
361-
self.data = xp.load(path)
361+
self.data = xp.load(str(path))
362362

363363
serializers = OrderedDict((
364364
('key2row', load_key2row),

0 commit comments

Comments
 (0)