Skip to content

Fix Unicode character error building Japanese PDF documents #145

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 0 commits into from
Closed

Fix Unicode character error building Japanese PDF documents #145

wants to merge 0 commits into from

Conversation

atsuoishimoto
Copy link
Contributor

Fix Unicode character error building Japanese PDF documents.

See python/python-docs-ja#31, python/python-docs-ja#35

  • Use LuaTex instead of pLatex.
  • use ltjsbook and ltjsarticle as document class.

@atsuoishimoto atsuoishimoto changed the title [WIP] Fix Unicode character error building Japanese PDF documents Fix Unicode character error building Japanese PDF documents Jan 17, 2023
@atsuoishimoto atsuoishimoto mentioned this pull request Jan 17, 2023
Closed
ezio-melotti
ezio-melotti reviewed Jan 17, 2023
View reviewed changes
build_docs.py Outdated
@@ -226,9 +226,11 @@ class Language:
)

PLATEX_DEFAULT = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
PLATEX_DEFAULT = (
LUALATEX_DEFAULT = (

Should the variable name be updated too?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I updated in #149.

build_docs.py Outdated
@@ -257,7 +259,7 @@ class Language:
Language("fr", "fr", "French", True, XELATEX_WITH_FONTSPEC),
Language("id", "id", "Indonesian", False, XELATEX_DEFAULT),
Language("it", "it", "Italian", False, XELATEX_DEFAULT),
Language("ja", "ja", "Japanese", True, PLATEX_DEFAULT, html_only=True), # See https://github.com/python/python-docs-ja/issues/35
Language("ja", "ja", "Japanese", True, PLATEX_DEFAULT, html_only=True),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Language("ja", "ja", "Japanese", True, PLATEX_DEFAULT, html_only=True),
Language("ja", "ja", "Japanese", True, LUALATEX_DEFAULT, html_only=True),

Also later in the file there's a piece of code that prints the platex version, that should be updated to print the lualatex version instead if platex is no longer used anywhere.

@JulienPalard
Copy link
Member

If I undertand correctly this PR is stuck with the newlines in the preamble?

@JulienPalard
Copy link
Member

As a「 « "temporary" » 」fix, would running:

sed -i $'s/\xef\xbf\xbd/\xef\xa3\xbd/g' Doc/**/*.rst

help? It could be easily added to docsbuild script inside an if language.tag == "ja" and with a big comment explaining why.

@atsuoishimoto
Copy link
Contributor Author

Sorry, I had to start over with a new branch. Please take a look at #149.

@atsuoishimoto
Copy link
Contributor Author

sed -i $'s/\xef\xbf\xbd/\xef\xa3\xbd/g' Doc/**/*.rst

With this change, the pdf is fine. But unfortunately, the HTML document will be messed up. I gave up to support U+FFFD with luatex(for now). Please refer #149.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ezio-melotti ezio-melotti ezio-melotti left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@atsuoishimoto @JulienPalard @ezio-melotti