Skip to content

Commit bfbac12

Browse files
lemirelemire
authored
We were forgetting to check the end bytes at the end of the UTF8 validation. (simdjson#1173)
* We were forgetting to check the end bytes at the end of the UTF8 validation. * Silencing the sanitizer * Better explanation.
1 parent 461f7dc commit bfbac12

11 files changed

+54
-1511
lines changed

‎src/generic/stage1/buf_block_reader.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,6 +75,7 @@ simdjson_really_inline const uint8_t *buf_block_reader<STEP_SIZE>::full_block()
7575

7676
template<size_t STEP_SIZE>
7777
simdjson_really_inline size_t buf_block_reader<STEP_SIZE>::get_remainder(uint8_t *dst) const {
78+
if(len == idx) { return 0; } // memcpy(dst, null, 0) will trigger an error with some sanitizers
7879
memset(dst, 0x20, STEP_SIZE); // memset STEP_SIZE because it's more efficient to write out 8 or 16 bytes at once.
7980
memcpy(dst, buf + idx, len - idx);
8081
return len - idx;

‎src/generic/stage1/json_structural_indexer.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -221,6 +221,7 @@ simdjson_really_inline error_code json_structural_indexer::finish(dom_parser_imp
221221
}
222222
parser.n_structural_indexes = new_structural_indexes;
223223
}
224+
checker.check_eof();
224225
return checker.errors();
225226
}
226227

‎src/generic/stage1/utf8_fastvalidate_algorithm.h

Lines changed: 0 additions & 184 deletions
This file was deleted.

0 commit comments

Comments
 (0)