gh-130703: Implement wrapping to width for msgids #130705
Conversation
|
Requesting @tomasr8 @serhiy-storchaka :-) |
|
I need to update normalize to wrap respecting words |
|
I'm afraid textwrap won't always work. I suggest adding the wrapping logic to the normalize function. pybabel does it in a similar way, you can have a look at their implementation: https://github.com/python-babel/babel/blob/master/babel/messages/pofile.py#L464 |
|
Implemented pybabels method. |
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
|
I really recommend creating a dummy file with some gettext calls and comparing the differences between pygettext, xgettext and babel. There are some differences that should be considered. Here's two I noticed:
_('foos')ran with msgid ""
""
"foos"
msgstr ""while xgettext and babel give me this (i.e. they don't insert two extra msgid "foos"
msgstr "" |
|
As for the header, this will conflict with my implementation of |
|
Test fail unrelated. Wrapping header will require a separate function like so: Subject: [PATCH] Wrap header
---
Index: Tools/i18n/pygettext.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/Tools/i18n/pygettext.py b/Tools/i18n/pygettext.py
--- a/Tools/i18n/pygettext.py (revision 8d03cbf141068c4ac9812a967a4c9f5942e22d75)
+++ b/Tools/i18n/pygettext.py (date 1740913002600)
@@ -589,12 +589,36 @@
def _is_string_const(self, node):
return isinstance(node, ast.Constant) and isinstance(node.value, str)
+
+def _wrap_header(s, options):
+ lines = []
+ for line in s.splitlines():
+ if len(line) > options.width and ' ' in line:
+ words = _space_splitter(line)
+ words.reverse()
+ buf = []
+ size = 0
+ while words:
+ word = words.pop()
+ if size + len(word) <= options.width:
+ buf.append(word)
+ size += len(word)
+ else:
+ lines.append(''.join(buf))
+ buf = [word]
+ size = len(word)
+ lines.append(''.join(buf))
+ else:
+ lines.append(line)
+ return "\n".join(lines) + "\n"
+
+
def write_pot_file(messages, options, fp):
timestamp = time.strftime('%Y-%m-%d %H:%M%z')
encoding = fp.encoding if fp.encoding else 'UTF-8'
- print(pot_header % {'time': timestamp, 'version': __version__,
+ print(_wrap_header(pot_header % {'time': timestamp, 'version': __version__,
'charset': encoding,
- 'encoding': '8bit'}, file=fp)
+ 'encoding': '8bit'}, options), file=fp)
# Sort locations within each message by filename and lineno
sorted_keys = [ |
There was a problem hiding this comment.
It looks almost ready now. Please add more tests for cases. It may be convenient to use the same string with different widths.
Add tests for the cases when len(escaped_line) + len(prefix) + 3 equals to width and when it equals to width.
Add tests for the cases when new_size + 2 equals to width and when it equals to width + 1.
Add tests for too long first word (new_size + 2 > widthandbuf` is empty) and for too long last word.
Add tests for whitespaces other than ' ' and '\n' (e.g. for '\t' and '\r'), for non-ASCII line separators and whitespaces. Test for different escaping mode.
Do not add a separate method for every case. Group assertions for similar cases in one method.
|
Friendly ping @serhiy-storchaka :-) |
|
Sorry, the tests still do not satisfy me. I am going to play with them myself, and then propose my variant. |
widthis not implemented for msgids #130703