Skip to content

Commit e88ca6c

Browse files
yzhangcsyzhangcs
committed
Align to main
1 parent d712165 commit e88ca6c

File tree

15 files changed

+198
-199
lines changed

15 files changed

+198
-199
lines changed

‎EXAMPLES.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ $ python -u -m supar.cmds.biaffine_dep train -b -d 0 -c biaffine-dep-en -p model
1515
--dev ptb/dev.conllx \
1616
--test ptb/test.conllx \
1717
--embed glove.6B.100d.txt \
18-
--unk
18+
--unk unk
1919
# crf2o
2020
$ python -u -m supar.cmds.crf2o_dep train -b -d 0 -c crf2o-dep-en -p model -f char \
2121
--train ptb/train.conllx \
@@ -30,7 +30,7 @@ The option `-c` controls where to load predefined configs, you can either specif
3030
For CRF models, you need to specify `--proj` to remove non-projective trees.
3131
Specifying `--mbr` to perform MBR decoding often leads to consistent improvement.
3232

33-
The model finetuned on [`robert-large`](https://huggingface.co/roberta-large) achieves nearly state-of-the-art performance in English dependency parsing.
33+
The model trained by finetuning [`robert-large`](https://huggingface.co/roberta-large) achieves nearly state-of-the-art performance in English dependency parsing.
3434
Here we provide some recommended hyper-parameters (not the best, but good enough).
3535
You are allowed to set values of registered/unregistered parameters in bash to suppress default configs in the file.
3636
```sh
@@ -46,7 +46,7 @@ $ python -u -m supar.cmds.biaffine_dep train -b -d 0 -c biaffine-dep-roberta-en
4646
--epochs=10 \
4747
--update-steps=4
4848
```
49-
The pretrained multilingual model `biaffine-dep-xlmr` takes [`xlm-roberta-large`](https://huggingface.co/xlm-roberta-large) as backbone architecture and finetunes on it.
49+
The pretrained multilingual model `biaffine-dep-xlmr` takes [`xlm-roberta-large`](https://huggingface.co/xlm-roberta-large) as backbone architecture and finetunes it.
5050
The training command is as following:
5151
```sh
5252
$ python -u -m supar.cmds.biaffine_dep train -b -d 0 -c biaffine-dep-xlmr -p model \

‎README.md

Lines changed: 33 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ All results are tested on the machine with Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.2
4545

4646
English and Chinese dependency parsing models are trained on PTB and CTB7 respectively.
4747
For each parser, we provide pretrained models that take BiLSTM as encoder.
48-
We also provide models finetuned on pretrained language models from [Huggingface Transformers](https://github.com/huggingface/transformers).
48+
We also provide models trained by finetuning pretrained language models from [Huggingface Transformers](https://github.com/huggingface/transformers).
4949
We use [`robert-large`](https://huggingface.co/roberta-large) for English and [`hfl/chinese-electra-180g-large-discriminator`](https://huggingface.co/hfl/chinese-electra-180g-large-discriminator) for Chinese.
5050
During evaluation, punctuation is ignored in all metrics for PTB.
5151

@@ -110,14 +110,14 @@ The results of each treebank are as follows.
110110
English semantic dependency parsing models are trained on [DM data introduced in SemEval-2014 task 8](https://catalog.ldc.upenn.edu/LDC2016T10), while Chinese models are trained on [NEWS domain data of corpora from SemEval-2016 Task 9](https://github.com/HIT-SCIR/SemEval-2016).
111111
Our data preprocessing steps follow [Second_Order_SDP](https://github.com/wangxinyu0922/Second_Order_SDP).
112112

113-
| Name | P | R | F<sub>1 | Sents/s |
114-
| ------------------------- | :---: | :---: | :-----: | ------: |
115-
| `biaffine-sdp-en` | 94.35 | 93.12 | 93.73 | 1067.06 |
116-
| `vi-sdp-en` | 94.36 | 93.52 | 93.94 | 821.73 |
117-
| `biaffine-sdp-roberta-en` | 95.07 | 95.22 | 95.15 | 269.05 |
118-
| `biaffine-sdp-zh` | 72.93 | 66.29 | 69.45 | 523.36 |
119-
| `vi-sdp-zh` | 72.05 | 67.97 | 69.95 | 411.94 |
120-
| `biaffine-sdp-electra-zh` | 71.49 | 70.08 | 70.78 | 143.04 |
113+
| Name | P | R | F<sub>1 | Sents/s |
114+
| ------------------- | :---: | :---: | :-----: | ------: |
115+
| `biaffine-sdp-en` | 94.35 | 93.12 | 93.73 | 1067.06 |
116+
| `vi-sdp-en` | 94.36 | 93.52 | 93.94 | 821.73 |
117+
| `vi-sdp-roberta-en` | 95.18 | 95.20 | 95.19 | 264.13 |
118+
| `biaffine-sdp-zh` | 72.93 | 66.29 | 69.45 | 523.36 |
119+
| `vi-sdp-zh` | 72.05 | 67.97 | 69.95 | 411.94 |
120+
| `vi-sdp-electra-zh` | 73.29 | 70.53 | 71.89 | 139.52 |
121121

122122
## Usage
123123

@@ -152,12 +152,13 @@ probs: tensor([1.0000, 0.9999, 0.9966, 0.8944, 1.0000, 1.0000, 0.9999])
152152
```
153153

154154
`SuPar` also supports parsing from tokenized sentences or from file.
155-
For semantic dependency parsing, lemmas and POS tags are needed.
155+
For BiLSTM-based semantic dependency parsing models, lemmas and POS tags are needed.
156156

157157
```py
158158
>>> import os
159159
>>> import tempfile
160-
>>> Parser.load('biaffine-dep-en').predict(['I','saw','Sarah','with','a','telescope','.'], verbose=False)[0]
160+
>>> dep = Parser.load('biaffine-dep-en')
161+
>>> dep.predict(['I', 'saw', 'Sarah', 'with', 'a', 'telescope', '.'], verbose=False)[0]
161162
1 I _ _ _ _ 2 nsubj _ _
162163
2 saw _ _ _ _ 0 root _ _
163164
3 Sarah _ _ _ _ 2 dobj _ _
@@ -185,7 +186,7 @@ For semantic dependency parsing, lemmas and POS tags are needed.
185186
186187
''')
187188
...
188-
>>> Parser.load('biaffine-dep-en').predict(path, pred='pred.conllx', verbose=False)[0]
189+
>>> dep.predict(path, pred='pred.conllx', verbose=False)[0]
189190
# text = But I found the location wonderful and the neighbors very kind.
190191
1 But _ _ _ _ 3 cc _ _
191192
2 I _ _ _ _ 3 nsubj _ _
@@ -201,13 +202,26 @@ For semantic dependency parsing, lemmas and POS tags are needed.
201202
11 kind _ _ _ _ 6 conj _ _
202203
12 . _ _ _ _ 3 punct _ _
203204

204-
>>> Parser.load('crf-con-en').predict(['I','saw','Sarah','with','a','telescope','.'], verbose=False)[0]
205-
(TOP (S (NP (_ I)) (VP (_ saw) (NP (_ Sarah)) (PP (_ with) (NP (_ a) (_ telescope)))) (_ .)))
206-
>>> Parser.load('biaffine-sdp-en').predict([[('I','I','PRP'), ('saw','see','VBD'),
207-
('Sarah','Sarah','NNP'), ('with','with','IN'),
208-
('a','a','DT'), ('telescope','telescope','NN'),
209-
('.','_','.')]],
210-
verbose=False)[0]
205+
>>> con = Parser.load('crf-con-en')
206+
>>> con.predict(['I', 'saw', 'Sarah', 'with', 'a', 'telescope', '.'], verbose=False)[0].pretty_print()
207+
TOP
208+
|
209+
S
210+
_____________|______________________
211+
| VP |
212+
| _________|____ |
213+
| | | PP |
214+
| | | ____|___ |
215+
NP | NP | NP |
216+
| | | | ___|______ |
217+
_ _ _ _ _ _ _
218+
| | | | | | |
219+
I saw Sarah with a telescope .
220+
221+
>>> sdp = Parser.load('biaffine-sdp-en')
222+
>>> sdp.predict([[('I','I','PRP'), ('saw','see','VBD'), ('Sarah','Sarah','NNP'), ('with','with','IN'),
223+
('a','a','DT'), ('telescope','telescope','NN'), ('.','_','.')]],
224+
verbose=False)[0]
211225
1 I I PRP _ _ _ _ 2:ARG1 _
212226
2 saw see VBD _ _ _ _ 0:root|4:ARG1 _
213227
3 Sarah Sarah NNP _ _ _ _ 2:ARG2 _

‎setup.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
setup(
66
name='supar',
7-
version='1.1.0',
7+
version='1.1.1',
88
author='Yu Zhang',
99
author_email='yzhang.cs@outlook.com',
1010
description='Syntactic/Semantic Parsing Models',
@@ -27,6 +27,7 @@
2727
'transformers>=4.0.0',
2828
'nltk',
2929
'stanza',
30+
'opt_einsum',
3031
'dill'],
3132
entry_points={
3233
'console_scripts': [

‎supar/__init__.py

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44
BiaffineSemanticDependencyParser, CRF2oDependencyParser,
55
CRFConstituencyParser, CRFDependencyParser, Parser,
66
VIConstituencyParser, VIDependencyParser,
7-
VISemanticDependencyParser, VISemanticRoleLabelingParser)
7+
VISemanticDependencyParser)
88

99
__all__ = ['BiaffineDependencyParser',
1010
'CRFDependencyParser',
@@ -14,10 +14,9 @@
1414
'VIConstituencyParser',
1515
'BiaffineSemanticDependencyParser',
1616
'VISemanticDependencyParser',
17-
'VISemanticRoleLabelingParser',
1817
'Parser']
1918

20-
__version__ = '1.1.0'
19+
__version__ = '1.1.1'
2120

2221
PARSER = {parser.NAME: parser for parser in [BiaffineDependencyParser,
2322
CRFDependencyParser,
@@ -26,11 +25,10 @@
2625
CRFConstituencyParser,
2726
VIConstituencyParser,
2827
BiaffineSemanticDependencyParser,
29-
VISemanticDependencyParser,
30-
VISemanticRoleLabelingParser]}
31-
32-
SRC = 'http://hlt.suda.edu.cn/LA/yzhang/supar'
28+
VISemanticDependencyParser]}
3329

30+
SRC = {'github': 'https://github.com/yzhangcs/parser/releases/download',
31+
'hlt': 'http://hlt.suda.edu.cn/LA/yzhang/supar'}
3432
NAME = {
3533
'biaffine-dep-en': 'ptb.biaffine.dep.lstm.char',
3634
'biaffine-dep-zh': 'ctb7.biaffine.dep.lstm.char',
@@ -48,9 +46,8 @@
4846
'biaffine-sdp-zh': 'semeval16.biaffine.sdp.lstm.tag-char-lemma',
4947
'vi-sdp-en': 'dm.vi.sdp.lstm.tag-char-lemma',
5048
'vi-sdp-zh': 'semeval16.vi.sdp.lstm.tag-char-lemma',
51-
'biaffine-sdp-roberta-en': 'dm.biaffine.sdp.roberta',
52-
'biaffine-sdp-electra-zh': 'semeval16.biaffine.sdp.electra'
49+
'vi-sdp-roberta-en': 'dm.vi.sdp.roberta',
50+
'vi-sdp-electra-zh': 'semeval16.vi.sdp.electra'
5351
}
54-
55-
MODEL = {n: f'{SRC}/v{__version__}/{m}.zip' for n, m in NAME.items()}
56-
CONFIG = {n: f'{SRC}/v{__version__}/{m}.ini' for n, m in NAME.items()}
52+
MODEL = {n: f"{SRC['github']}/v1.1.0/{m}.zip" for n, m in NAME.items()}
53+
CONFIG = {n: f"{SRC['github']}/v1.1.0/{m}.ini" for n, m in NAME.items()}

‎supar/cmds/cmd.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,6 @@ def parse(parser):
1212
parser.add_argument('--device', '-d', default='-1', help='ID of GPU to use')
1313
parser.add_argument('--seed', '-s', default=1, type=int, help='seed for generating random numbers')
1414
parser.add_argument('--threads', '-t', default=16, type=int, help='max num of threads')
15-
parser.add_argument('--batch-size', default=5000, type=int, help='batch size')
1615
parser.add_argument("--local_rank", type=int, default=-1, help='node rank for distributed training')
1716
args, unknown = parser.parse_known_args()
1817
args, unknown = parser.parse_known_args(unknown, args)

‎supar/models/__init__.py

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
CRFDependencyModel, VIDependencyModel)
66
from .model import Model
77
from .sdp import BiaffineSemanticDependencyModel, VISemanticDependencyModel
8-
from .srl import VISemanticRoleLabelingModel
98

109
__all__ = ['Model',
1110
'BiaffineDependencyModel',
@@ -15,5 +14,4 @@
1514
'CRFConstituencyModel',
1615
'VIConstituencyModel',
1716
'BiaffineSemanticDependencyModel',
18-
'VISemanticDependencyModel',
19-
'VISemanticRoleLabelingModel']
17+
'VISemanticDependencyModel']

‎supar/modules/affine.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
import torch
44
import torch.nn as nn
5+
from opt_einsum import contract
56

67

78
class Biaffine(nn.Module):
@@ -71,7 +72,7 @@ def forward(self, x, y):
7172
if self.bias_y:
7273
y = torch.cat((y, torch.ones_like(y[..., :1])), -1)
7374
# [batch_size, n_out, seq_len, seq_len]
74-
s = torch.einsum('bxi,oij,byj->boxy', x, self.weight, y) / self.n_in ** self.scale
75+
s = contract('bxi,oij,byj->boxy', x, self.weight, y) / self.n_in ** self.scale
7576
# remove dim 1 if n_out == 1
7677
s = s.squeeze(1)
7778

@@ -145,9 +146,9 @@ def forward(self, x, y, z):
145146
x = torch.cat((x, torch.ones_like(x[..., :1])), -1)
146147
if self.bias_y:
147148
y = torch.cat((y, torch.ones_like(y[..., :1])), -1)
148-
w = torch.einsum('bzk,oikj->bozij', z, self.weight)
149+
w = contract('bzk,oikj->bozij', z, self.weight)
149150
# [batch_size, n_out, seq_len, seq_len, seq_len]
150-
s = torch.einsum('bxi,bozij,byj->bozxy', x, w, y) / self.n_in ** self.scale
151+
s = contract('bxi,bozij,byj->bozxy', x, w, y) / self.n_in ** self.scale
151152
# remove dim 1 if n_out == 1
152153
s = s.squeeze(1)
153154

‎supar/parsers/__init__.py

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@
55
CRFDependencyParser, VIDependencyParser)
66
from .parser import Parser
77
from .sdp import BiaffineSemanticDependencyParser, VISemanticDependencyParser
8-
from .srl import VISemanticRoleLabelingParser
98

109
__all__ = ['BiaffineDependencyParser',
1110
'CRFDependencyParser',
@@ -15,5 +14,4 @@
1514
'VIConstituencyParser',
1615
'BiaffineSemanticDependencyParser',
1716
'VISemanticDependencyParser',
18-
'VISemanticRoleLabelingParser',
1917
'Parser']

‎supar/parsers/con.py

Lines changed: 19 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
from supar.models import CRFConstituencyModel, VIConstituencyModel
88
from supar.parsers.parser import Parser
99
from supar.utils import Config, Dataset, Embedding
10-
from supar.utils.common import bos, eos, pad, unk
10+
from supar.utils.common import BOS, EOS, PAD, UNK
1111
from supar.utils.field import ChartField, Field, RawField, SubwordField
1212
from supar.utils.logging import get_logger, progress_bar
1313
from supar.utils.metric import SpanMetric
@@ -129,7 +129,7 @@ def predict(self, data, pred=None, lang=None, buckets=8, batch_size=5000, prob=F
129129
return super().predict(**Config().update(locals()))
130130

131131
@classmethod
132-
def load(cls, path, reload=False, **kwargs):
132+
def load(cls, path, reload=False, src=None, **kwargs):
133133
r"""
134134
Loads a parser with data fields and pretrained model parameters.
135135
@@ -140,6 +140,11 @@ def load(cls, path, reload=False, **kwargs):
140140
- a local path to a pretrained model, e.g., ``./<path>/model``.
141141
reload (bool):
142142
Whether to discard the existing cache and force a fresh download. Default: ``False``.
143+
src (str):
144+
Specifies where to download the model.
145+
``'github'``: github release page.
146+
``'hlt'``: hlt homepage, only accessible from 9:00 to 18:00 (UTC+8).
147+
Default: None.
143148
kwargs (dict):
144149
A dict holding unconsumed arguments for updating training configs and initializing the model.
145150
@@ -149,7 +154,7 @@ def load(cls, path, reload=False, **kwargs):
149154
>>> parser = Parser.load('./ptb.crf.con.lstm.char')
150155
"""
151156

152-
return super().load(path, reload, **kwargs)
157+
return super().load(path, reload, src, **kwargs)
153158

154159
def _train(self, loader):
155160
self.model.train()
@@ -246,7 +251,7 @@ def build(cls, path, min_freq=2, fix_len=20, **kwargs):
246251
return parser
247252

248253
logger.info("Building the fields")
249-
WORD = Field('words', pad=pad, unk=unk, bos=bos, eos=eos, lower=True)
254+
WORD = Field('words', pad=PAD, unk=UNK, bos=BOS, eos=EOS, lower=True)
250255
TAG, CHAR, BERT = None, None, None
251256
if args.encoder != 'lstm':
252257
from transformers import (AutoTokenizer, GPT2Tokenizer,
@@ -262,11 +267,11 @@ def build(cls, path, min_freq=2, fix_len=20, **kwargs):
262267
fn=None if not isinstance(t, (GPT2Tokenizer, GPT2TokenizerFast)) else lambda x: ' '+x)
263268
WORD.vocab = t.get_vocab()
264269
else:
265-
WORD = Field('words', pad=pad, unk=unk, bos=bos, eos=eos, lower=True)
270+
WORD = Field('words', pad=PAD, unk=UNK, bos=BOS, eos=EOS, lower=True)
266271
if 'tag' in args.feat:
267-
TAG = Field('tags', bos=bos, eos=eos)
272+
TAG = Field('tags', bos=BOS, eos=EOS)
268273
if 'char' in args.feat:
269-
CHAR = SubwordField('chars', pad=pad, unk=unk, bos=bos, eos=eos, fix_len=args.fix_len)
274+
CHAR = SubwordField('chars', pad=PAD, unk=UNK, bos=BOS, eos=EOS, fix_len=args.fix_len)
270275
if 'bert' in args.feat:
271276
from transformers import (AutoTokenizer, GPT2Tokenizer,
272277
GPT2TokenizerFast)
@@ -411,7 +416,7 @@ def predict(self, data, pred=None, lang=None, buckets=8, batch_size=5000, prob=F
411416
return super().predict(**Config().update(locals()))
412417

413418
@classmethod
414-
def load(cls, path, reload=False, **kwargs):
419+
def load(cls, path, reload=False, src=None, **kwargs):
415420
r"""
416421
Loads a parser with data fields and pretrained model parameters.
417422
@@ -422,6 +427,11 @@ def load(cls, path, reload=False, **kwargs):
422427
- a local path to a pretrained model, e.g., ``./<path>/model``.
423428
reload (bool):
424429
Whether to discard the existing cache and force a fresh download. Default: ``False``.
430+
src (str):
431+
Specifies where to download the model.
432+
``'github'``: github release page.
433+
``'hlt'``: hlt homepage, only accessible from 9:00 to 18:00 (UTC+8).
434+
Default: None.
425435
kwargs (dict):
426436
A dict holding unconsumed arguments for updating training configs and initializing the model.
427437
@@ -431,7 +441,7 @@ def load(cls, path, reload=False, **kwargs):
431441
>>> parser = Parser.load('./ptb.vi.con.lstm.char')
432442
"""
433443

434-
return super().load(path, reload, **kwargs)
444+
return super().load(path, reload, src, **kwargs)
435445

436446
def _train(self, loader):
437447
self.model.train()

0 commit comments

Comments
 (0)