feat: json-records content type support #167
Conversation
|
|
src/model/events/body-formatting.ts
Outdated
| @@ -127,6 +127,40 @@ export const Formatters: { [key in ViewableContentType]: Formatter } = { | |||
| } | |||
| } | |||
| }, | |||
| 'json-records': { | |||
| language: 'json', | |||
There was a problem hiding this comment.
With this approach, we're parsing & converting the JSON records to a sort-of equivalent JSON array, and then just showing that as JSON content.
This sort of works but it would be better to create a separate language I think. There's a few problems here I can see already:
- It's not an accurate representation of the content - it looks like you've received an array
[ ... ]but you haven't really. - When you switch between this & normal JSON format on the same content, the formatting changes but the errors don't seem to reset (although, as in my other comment, we can fix this by just making JSON not appear in the dropdown) I think because they share the same
language - Parsing & validation doesn't work properly - any parsing errors result in the content collapsing back to an
{}�{{...unformatted string, which then just shows errors for the record separator characters (not the actual error in the content). We should be able to split & format the messages regardless of JSON parsing, and show nice errors inline.
Did you take a look at the XML support code? I think the same thing should be possible. If you want to look at Monaco's code for JSON support itself, the entrypoint is here.
There was a problem hiding this comment.
Think the JSON-records is a kind of variant of JSON, so I reused the json language. Let me try to update this with a separate language
There was a problem hiding this comment.
Seems the jsonc-parser did not support return the error line number/column number info, any suggestion?
https://github.com/microsoft/node-jsonc-parser/blob/7eba2392a9aecb76fc150705454ce277dc7b7f6b/src/impl/parser.ts#L162
https://github.com/microsoft/node-jsonc-parser/blob/main/src/main.ts#L143
There was a problem hiding this comment.
Good point, I thought it did, and it has the internals but it doesn't expose that as you'd expect. I've just opened a PR to fix that jsonc-parser: microsoft/node-jsonc-parser#102.
No way to know how long that'll take to merge & release though, so better to assume we don't have it in the meantime. It's easy to work around though: you can copy the definition of parse in directly (from my PR there, so it includes the error line details) since it's just a small wrapper around jsoncParser.visit. Once/if the PR above is merged we can drop that and use jsoncParser.parse directly instead.
This now uses parseTree with model.getPosition to get the exact correct line & column for errors, and to handle multiple errors in the same line or elsewhere, and accepts JSON lines & NDJSON in addition to 0x1E separated records like SignalR. Detection is extended to match, and uses some slightly broader heuristics to catch a few more cases.
|
Thanks @WeihanLi! I've added a few commits on top of this: the first two commits here are really just reorganizing a little and using jsonc-parser more effectively so that we can get accurate line & column numbers on errors, even in the tricky edge cases, and to support newline separators as well. The third commit I went a bit over the top - after playing with jsonc-parser, I realised we could use the scanner to completely change how we do JSON formatting for everything (records and elsewhere) so it works nicely even with invalid content, and gives us much a cleaner view of everything in lots of cases. I'm sneaking it here since I think it'll work really nicely with this JSON records support. I've added some tests and done a bit of my own checking with manual values, but if you have a minute it'd be really useful if you could pull the result, and test it against your real use case so we can confirm it does nicely solve the original problem. |
Looks great to me, thanks very much @pimterry
|
|
Really glad we could get this fixed @WeihanLi! This should be a great improvement for quite a few cases I think. As I mentioned, all contributors get free HTTP Toolkit Pro - I've just set up an account for you with the email from your git commits, just click 'Get Pro' then 'Log into existing account' and enter your email to get started. |
|
@pimterry thanks very much for your help and the amazing tool! |
fixes httptoolkit/httptoolkit#772
This pull request introduces support for a new
json-recordscontent type, allowing for the parsing and formatting of JSON records separated by specific delimiters. Key changes include updates to content type handling, new utility functions, and enhancements to formatters for synchronous and asynchronous processing.Support for
json-recordscontent type:json-recordsas a newViewableContentTypeinsrc/model/events/content-types.ts, with associated mappings and logic for determining compatibility based on content structure. [1] [2] [3] [4]StreamMessageclass to identifyjson-recordscontent based on specific delimiters. [1] [2]Enhancements to formatters:
json-recordsinsrc/model/events/body-formatting.tsto handle both synchronous and asynchronous parsing based on input size.json-recordsformatter tosrc/services/ui-worker-formatters.tsfor asynchronous processing in worker threads.Utility function for buffer splitting:
splitBufferinsrc/util/buffer.tsto split aBufferinto chunks using a specified separator, enabling efficient processing of JSON records.