Skip to content

Commit a2e3797

Browse files
committed
Fix style
1 parent a0dfc2d commit a2e3797

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

src/transformers/tokenization_utils_base.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3054,11 +3054,12 @@ def _pad(
30543054

30553055
def convert_tokens_to_string(self, tokens: List[str]) -> str:
30563056
"""
3057-
Converts a sequence of tokens in a single string. The most simple way to do it is ``" ".join(tokens)`` but
3058-
we often want to remove sub-word tokenization artifacts at the same time.
3057+
Converts a sequence of tokens in a single string. The most simple way to do it is ``" ".join(tokens)`` but we
3058+
often want to remove sub-word tokenization artifacts at the same time.
30593059
30603060
Args:
30613061
tokens (:obj:`List[str]`): The token to join in a string.
3062+
30623063
Returns:
30633064
:obj:`str`: The joined tokens.
30643065
"""

0 commit comments

Comments
 (0)