Update rustpython_literal::escape to support wtf8 #5630
Conversation
|
How about moving wtf8 to project root? When it is placed under common, it looks like only common can directly use wtf8 but nothing other |
|
Hmm, fair. |
|
At some point we should probably restructure to everything being under a |
| /// # Safety | ||
| /// | ||
| /// This string must only contain printable characters. | ||
| unsafe fn write_source(&self, formatter: &mut impl std::fmt::Write) -> std::fmt::Result; |
There was a problem hiding this comment.
I am confused about this change.
I expect UnicodeEscape::new_repr(self.as_wtf8()) always safely print the repr. Then the surrogate must be checked somewhere before printing. Doesn't it mean we always print &str, but not wtf8?
There was a problem hiding this comment.
We always print str, yes. I did this because we were already treating write_source as an unsafe function, per the preëxisting comment in AsciiEscape::write_source:
// SAFETY: this function must be called only when source is printable ascii charactersThat was technically unsound, since anybody could just call AsciiEscape::new_repr(b"\xff").write_source() and invoke library UB.
The first commit splits out
rustpython_common::wtf8into its own crate, so that things like literal and sre_engine don't have to depend on all of common just to implement something for wtf8. I also simplified the crate graph a little by removingcompiler's dependency oncommon, sinceexpandtabwas the only function left thatcompiler-codegenneeded, and that's simple enough to just copy over. Originally the point ofcommonwas for there to be shared code between vm and compiler, but I guess the times change :)Workspace crate graph before:
Workspace crate graph after: