@@ -167,9 +167,8 @@ Extractive Question Answering
167
167
168
168
Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
169
169
question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune a
170
- model on a SQuAD task, you may leverage the `run_squad.py
171
- <https://github.com/huggingface/transformers/tree/master/examples/question-answering/run_squad.py> `__ and
172
- `run_tf_squad.py
170
+ model on a SQuAD task, you may leverage the `run_qa.py
171
+ <https://github.com/huggingface/transformers/tree/master/examples/question-answering/run_qa.py> `__ and `run_tf_squad.py
173
172
<https://github.com/huggingface/transformers/tree/master/examples/question-answering/run_tf_squad.py> `__ scripts.
174
173
175
174
@@ -327,7 +326,9 @@ Masked language modeling is the task of masking tokens in a sequence with a mask
327
326
fill that mask with an appropriate token. This allows the model to attend to both the right context (tokens on the
328
327
right of the mask) and the left context (tokens on the left of the mask). Such a training creates a strong basis for
329
328
downstream tasks requiring bi-directional context, such as SQuAD (question answering, see `Lewis, Lui, Goyal et al.
330
- <https://arxiv.org/abs/1910.13461> `__, part 4.2).
329
+ <https://arxiv.org/abs/1910.13461> `__, part 4.2). If you would like to fine-tune a model on a masked language modeling
330
+ task, you may leverage the `run_mlm.py
331
+ <https://github.com/huggingface/transformers/tree/master/examples/language-modeling/run_mlm.py> `__ script.
331
332
332
333
Here is an example of using pipelines to replace a mask from a sequence:
333
334
@@ -435,7 +436,8 @@ Causal Language Modeling
435
436
436
437
Causal language modeling is the task of predicting the token following a sequence of tokens. In this situation, the
437
438
model only attends to the left context (tokens on the left of the mask). Such a training is particularly interesting
438
- for generation tasks.
439
+ for generation tasks. If you would like to fine-tune a model on a causal language modeling task, you may leverage the
440
+ `run_clm.py <https://github.com/huggingface/transformers/tree/master/examples/language-modeling/run_clm.py >`__ script.
439
441
440
442
Usually, the next token is predicted by sampling from the logits of the last hidden state the model produces from the
441
443
input sequence.
@@ -603,11 +605,7 @@ Named Entity Recognition (NER) is the task of classifying tokens according to a
603
605
as a person, an organisation or a location. An example of a named entity recognition dataset is the CoNLL-2003 dataset,
604
606
which is entirely based on that task. If you would like to fine-tune a model on an NER task, you may leverage the
605
607
`run_ner.py <https://github.com/huggingface/transformers/tree/master/examples/token-classification/run_ner.py >`__
606
- (PyTorch), `run_pl_ner.py
607
- <https://github.com/huggingface/transformers/tree/master/examples/token-classification/run_pl_ner.py> `__ (leveraging
608
- pytorch-lightning) or the `run_tf_ner.py
609
- <https://github.com/huggingface/transformers/tree/master/examples/token-classification/run_tf_ner.py> `__ (TensorFlow)
610
- scripts.
608
+ script.
611
609
612
610
Here is an example of using pipelines to do named entity recognition, specifically, trying to identify tokens as
613
611
belonging to one of 9 classes:
@@ -745,7 +743,9 @@ token. The following array should be the output:
745
743
Summarization
746
744
-----------------------------------------------------------------------------------------------------------------------
747
745
748
- Summarization is the task of summarizing a document or an article into a shorter text.
746
+ Summarization is the task of summarizing a document or an article into a shorter text. If you would like to fine-tune a
747
+ model on a summarization task, you may leverage the `run_seq2seq.py
748
+ <https://github.com/huggingface/transformers/tree/master/examples/seq2seq/run_seq2seq.py> `__ script.
749
749
750
750
An example of a summarization dataset is the CNN / Daily Mail dataset, which consists of long news articles and was
751
751
created for the task of summarization. If you would like to fine-tune a model on a summarization task, various
@@ -823,7 +823,9 @@ CNN / Daily Mail), it yields very good results.
823
823
Translation
824
824
-----------------------------------------------------------------------------------------------------------------------
825
825
826
- Translation is the task of translating a text from one language to another.
826
+ Translation is the task of translating a text from one language to another. If you would like to fine-tune a model on a
827
+ translation task, you may leverage the `run_seq2seq.py
828
+ <https://github.com/huggingface/transformers/tree/master/examples/seq2seq/run_seq2seq.py> `__ script.
827
829
828
830
An example of a translation dataset is the WMT English to German dataset, which has sentences in English as the input
829
831
data and the corresponding sentences in German as the target data. If you would like to fine-tune a model on a
0 commit comments